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1.  Introduction 


1.1  Motivation 

The  design  of  sensing  mechanisms  for  small  unmanned  aircraft  systems  (sUAS) 
has  many  tradeoffs  due  to  limited  budgets  for  power  consumption,  size,  weight,  and 
the  need  for  both  speed  and  accuracy  in  a  wide  range  of  operating  conditions. 
Traditionally,  inertial  measurement  units  (gyroscopes  and  accelerometers)  are  used 
to  obtain  velocity  and  position  data.  There  has  been  a  rapid  evolution  of  these  sensor 
systems  in  recent  years  toward  integrated  accelerometer  and  gyroscope  packages 
that  include  both  digitization  and  signal  conditioning  (e.g.,  filtering).  As  the  vehicle 
sizes  have  continued  to  decrease,  faster  sensing  is  needed  due  to  the  increased 
susceptibility  of  the  aircraft  to  tiniest  of  disturbances. 

Looking  to  nature,  several  species  of  flying  insects  have  been  demonstrated  to 
possess  exceptional  stability  and  acrobatic  capabilities  that  match  the  types  of 
missions  that  engineers  are  trying  to  accomplish.  They  provide  examples  of  robust 
stability  given  similar  limitations  of  sensing  and  processing.  The  insect  body  is  a 
multimodal  sensor  network.  Information  from  visual,  proprioceptive,  tactile,  and 
inertial  receptors  is  collected  to  provide  information  about  the  state  of  the  insect 
with  respect  to  its  environment.1  Instead  of  the  digital  architecture  used  in 
traditional  sUAS,  insects  have  analog  connections  between  their  sensory  systems 
and  their  flight  motor  neurons.  Analog  architecture  makes  them  capable  of  closing 
feedback  loops  at  high  speeds,  becoming  very  useful  for  fast  stabilization  for 
sudden  disturbances.  Bio-inspired  sensing  techniques  based  on  these  species 
present  an  attractive  way  for  microaerial-vehicle  sensor  design. 

Many  flying  insects  employ  2  visual  systems,  the  compound  eyes  and  the  ocelli 
(simple  eyes).  From  the  behavioral  and  electrophysiological  experiments  cited  in 
the  next  sections,  the  compound  eyes  and  ocelli  are  thought  to  work  together. 
Overall,  compound  eyes  are  sensitive  to  a  wide  range  of  information,  such  as 
proximity  to  obstacles,  relative  velocity,  and  rotation  rate.2  4  These  tasks  increase 
the  information-processing  time  for  the  compound  eyes,  making  them  unable  to 
provide  fast  responses  for  sudden  disturbances.  Insects  have  to  balance  themselves 
quickly  to  survive.  Ocelli,  responsible  for  a  fewer  number  of  tasks  compared  to 
compound  eyes,  have  less  processing  time,5,6  which  makes  them  favorable  to  detect 
sudden  disturbances.  Inspired  by  the  complementary  nature  of  ocelli  and  compound 
eyes,  this  report  attempts  to  characterize  the  frequency  response  of  an  ocellar  sensor 
and  optic  flow,  and  ultimately  proposes  the  fusion  of  2  sensors  for  low-cost,  wide- 
field,  visual  rotational  motion  sensing. 
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1.2  Contributions 


The  contributions  of  this  report  are  listed  as  follows: 

•  The  comparative  open-loop  frequency  characterization  of  optic  flow  and  a 
luminance-dependent  analog  rotation-rate  sensor  that  is  thought  to  mimic 
insect  ocelli  was  conducted. 

•  Sensitivity  analysis  was  done  to  analyze  the  parameters  that  affect  the  optic 
flow  and  ocellar  sensor  performance  in  rotational  motion. 

2.  Background 


2.1  Structure  and  Function  of  Compound  Eye 


The  compound  eyes  and  ocelli  are  shown  in  Fig.  1,  head  of  a  flying  insect  (Polistes). 
The  structure  of  compound  eyes  (large,  2  on  the  sides)  is  seen  in  Fig.  2.  The 
compound  eyes  are  composed  of  units  called  ommatidia.  Each  ommatidium  unit 
functions  as  a  separate  visual  receptor,  consisting  of  a  lens,  cornea,  a  crystalline 
cone,  light-sensitive  visual  cells,  and  pigment  cells  (Fig.  3).  There  may  be  up  to 
30,000  ommatidia  in  a  single  compound  eye.  The  image  perceived  is  a  combination 
of  inputs  from  ommatidia  pointing  at  slightly  different  directions  (as  seen  in  Fig.  2, 
ommatidia  units  make  up  a  conic  surface).  A  mosaic-like  vision  of  the  environment 
is  rendered.2,3 


Compound 

eye 

Ocelli 


Fig.  1  Insect  compound  eye  and  ocelli  (image  used  with  Wikimedia  Commons 
permissions:  https://c0mm0ns.wikimedia.0rg/wiki/File:P0listes_0celli.jpg) 
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Fig.  2  Structure  of  compound  eye  (image  used  with  Wikimedia  Commons  permissions: 
https://commons.wikimedia.Org/wiki/File:Insect_compound_eye_diagram.svg) 


-  Cornea  (facet) 


Crystalline  cone 
Upper  iris  cells 


Matrix  cells  of  cornea 


Lower  iris  cells 


Sensory  cells 


Axons  (form  optic 
nerves) 


Fig.  3  Structure  of  ommatidium  (image  used  with  permission  from  Cronodon.com: 

http  ://cronodon.com/  Copyright.html) 

Vision  process  starts  at  ommatidia.  Ommatidia  photoreceptors  capture  patterns  of 
luminance  from  the  visual  environment.  The  captured  signal  is  conditioned  through 
lamina  plate.  The  output  of  lamina  is  thought  to  be  the  input  to  medulla.7,8  The 
medulla  outputs  optic  flow-like  patterns  to  lobula,  and  lobula  processes  these 
outputs.9-13  The  output  signals  of  lamina  neurons  are  segregated  into  different 
pathways,  performing  functions  such  as  color  discrimination,  motion  detection,  and 
intensity  encoding.  Neurons  responding  to  motion  are  found  in  lobula.  They  are 
thought  to  receive  inputs  from  hypothetical  neural  elements  called  Reichardt 
Detectors,  or  elementary  motion  detectors  (EMDs),  residing  in  medulla  and 
calculating  motion  from  the  pixel-based  information  with  a  mechanism  called 
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Reichardt  correlation.14  This  hypothetical  mechanism  is  proposed  to  understand 
how  a  neuron,  which  is  only  receiving  luminance  input,  is  able  to  compute  motion. 
Frye.15  depicts  the  key  components  of  this  algorithm,  which  are  2  inputs  (red,  as 
photoreceptors),  a  time  delay  on  one  input  (d),  and  multiplication  on  correlated 
signals. 

1)  Photons  from  a  visual  scene  move  from  left  to  right. 

2)  Photons  activate  the  first  receptor. 

3)  The  signal  from  the  first  receptor  is  delayed  with  d  as  the  photons  move  to 
the  second  receptor. 

4)  Photons  activate  the  second  photoreceptor.  Both  the  delayed  signal  (from 
first  receptor)  and  the  undelayed  signal  (from  second  receptor)  converge 
simultaneously  onto  a  multiplication  stage,  producing  a  signal  related  to 
direction  of  motion.  Conversely,  photons  passing  from  right  to  left  will 
output  zero  for  the  opposite  direction,  since  there  is  no  delay  component 
that  will  deliver  simultaneous  inputs  to  multiplication  stage. 


2.2  Structure  and  Function  of  Ocelli 


Ocelli  differ  from  the  compound  eye  in  that  they  have  only  a  single  lens  covering 
an  array  of  photoreceptors,  as  seen  in  Fig.  4.  Ocelli  are  found  in  the  frontal  surface 
of  the  head  of  many  insects.  Ocelli  tend  to  be  larger  in  flying  insects  (bees, 
dragonflies,  locusts)  and  are  typically  found  as  a  triplet.  Two  lateral  ocelli  are  found 
in  the  left  and  right  of  the  head,  while  a  median  ocellus  is  directed  frontally. 


Fig.  4  Ocellus  cross  section  (image  used  with  Wikimedia  Commons  permissions: 
https://commons.wikimedia.Org/wiki/File:Insect_ocellus_diagram.svg;  see  Ref.  16) 
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Various  studies  have  been  conducted  to  reveal  the  function  of  ocelli  for  different 
insects.  Although  it  is  called  an  “eye”,  ocellus  is  claimed  to  be  underfocusing  the 
image,  hence  showing  hardly  any  image  details  for  a  locust.  In  contrast  to  the 
“underfocusing”  information  for  the  locust,17  suggests  that  the  dragonfly  ocellus, 
which  is  believed  to  be  highly  evolved,  is  able  to  detect  some  image  details. 

It  is  also  suggested  that  the  temporal  and  spatial  filtering  characteristics  of  locust 
ocelli  neurons  are  well  suited  to  detect  instability  in  flight.1819  The  stabilization  in 
flight  studies  were  summarized  by  Heisenberg  and  Reinhard,4  most  of  which 
studies  are  conducted  by  releasing  dragonflies  with  ablated  ocelli.  Dragonflies 
show  unstable  flight  attitudes  without  ocelli.  Kastberger  and  Schumann20  evaluate 
the  flight  behavior  of  bees  with  and  without  occluded  ocelli,  stating  that  normal 
bees  (without  occlusion)  show  quicker  flight  behaviors. 

Another  characteristic  of  ocelli  is  higher  photic  sensitivity,  compared  to  the 
compound  eye  for  locusts18  and  bees.21  This  information  is  useful  if  we  think  of  the 
ocelli  as  integrators  of  the  overall  intensity  or  a  blurred  visual  field.  If  the  photic 
sensitivity  is  high,  small  changes  in  light  intensity  will  be  sensed.  Taking  into 
account  that  the  images  sensed  by  the  ocelli  are  highly  blurred,  ocelli  should  be 
concerned  with  the  overall  image  intensity.  Studies  of  dragonflies22,23  and  locusts24 
claim  that  ocelli  are  rotation  detectors,  important  for  gaze  stabilization.  Research 
by  Schuppe  and  Hengstenberg25  also  shows  gaze  stabilization  cues  by  the  ocelli. 
The  beginning  and  end  of  daily  activities  of  insects  depend  on  light  intensity. 
Studies  of  bees,23  crickets,26  and  moths27-29  claim  that  the  ocelli  perceive  low  light 
intensity  to  control  daily  activities. 

Compared  to  ocelli,  compound  eyes  offer  a  panoramic  field  of  view  (FOV)  and 
high  temporal  resolution,  with  optic  flow  computation  abilities.30-32  These  features 
are  beneficial  for  tasks  like  visually  guided  navigation  (e.g.,  obstacle  avoidance, 
landing  strategy,  saccade  response,  hovering  strategy  clutter  response,  collision 
response,  and  fixation  strategy)  each  of  which  is  described  by  Barrows  et  al.33  with 
specific  test  setups  for  bees  and  drosophilae. 

2.3  Prior  Works  Inspired  by  Insect  Ocelli 

Because  of  the  prominent  computation  advantages,  simplicity,  and  applicability  to 
small-scale  world,  ocelli-inspired  vision  sensors  have  been  developed  by  many 
groups.  These  implementations  mainly  focus  on  closed-loop  attitude  control, 
outputting  pitch,  and  roll  angle.  Neumann  and  Biilthoff34  present  a  simulation 
model  of  an  autonomous  agent  flying  through  a  virtual  environment  with  a  daylight 
sky  model.  It  uses  over  200  receptors  to  detect  local  intensities.  These  receptors  are 
distributed  evenly  between  adjacent  directions  on  an  agent  body  coordinate  system. 
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The  average  intensity  difference  between  2  directions  is  computed  to  estimate  the 
roll  angle.  Subsequently,  a  simulation  of  an  eye  model  with  a  special  receptor 
distribution  was  presented  in  a  virtual  environment  in  their  2002  report.35  The 
ocelli-like  “wide-field  measurement  units”  that  use  a  locally  weighted  intensity  as 
receptor  response  are  subtracted  in  adjacent  directions.  Using  an  EMD  and  ocelli 
outputs,  optimal  receptive  fields  for  attitude  estimation,  yaw  rotations  and  nearness 
are  derived.  A  2003  study36  implements  ocelli,  haltere  (an  organ  responsible  for 
balance  in  flying  insects),  optic  flow,  and  magnetic  flow  sensors  for  a 
micromechanical  flying  insect.  These  sensors  were  used  to  estimate  body  attitude 
relative  to  a  fixed  frame,  body  rotational  velocities,  obstacle  avoidance,  and 
heading  adjustment,  respectively.  The  ocelli  consist  of  4  photodiodes,  arranged  in 
a  pyramidal  configuration.  The  2  output  signals  are  obtained  by  subtracting  the 
opposite  photodiode  outputs.  Schenato  et  al.37  use  this  implementation  and 
proposes  a  stabilizing  attitude  control  law  for  a  sinusoidal  intensity  function.  Javaan 
et  al.38  demonstrate  an  embedded  implementation  of  ocelli-like  sensor.  It  uses  the 
difference  between  ultraviolet  and  green  photodiode  signals  to  obtain  attitude 
estimation,  stating  that  this  reduces  the  biasing  effect  of  clouds  and  the  sun.  Kerhuel 
et  al.39  use  a  camera  to  track  a  reference  heading  point  and  perform  gaze 
stabilization  by  using  the  difference  between  reference  and  instantaneous  heading 
signal.  Moore  et  al.40  use  camera  images  that  are  classified  by  the  intensity 
information  in  the  YUV  (luminance,  blue,  red)  channel  into  sky  and  ground  regions 
to  estimate  roll  and  pitch  angles.  Javaan  and  Akiko41  use  4  ultraviolet/green 
photodiode  pairs  to  detect  attitude  angle  and  demonstrates  roll  attitude  tracking  on 
an  aircraft.  A  2014  study42  proposes  an  ocelli -based  sensor,  which  is  also  used  in 
this  work,  to  output  roll  and  pitch  rate,  rather  than  angle.  This  sensor  was  used  for 
the  frequency  characterization  in  Section  3  of  this  report. 

3.  Frequency-Domain  Characterization  of  Ocellar  Sensor  and 
Optic  Flow 


3.1  Introduction 

Gremillion  et  al.42  present  experimental  data  that  use  the  complementary  response 
of  an  analog  ocellar  sensor  and  a  commercial  optic  flow  sensor.  Inspired  by  this 
complementary  response  information  and  cues  from  insect  ocelli  and  compound 
eye  complementary  task  mechanism,  we  designed  a  test  platform  that  generates 
rotational  motion  to  characterize  the  frequency-domain  response  of  both  optic  flow 
and  an  ocellar  sensor,  and  gathers  information  from  different  sources  such  as  motor 
controller,  microcontroller,  and  gyroscope.  The  optic  flow  is  computed  using  the 
images  collected  by  a  camera  and  fisheye  lens.  A  microelectromechanical  systems 
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gyroscope  and  a  Vicon  motion-detection  system  are  used  as  ground  truth.  This 
section  discusses  the  ocelli  and  optic  flow  frequency  response  characteristics  and 
the  performance  parameters  for  the  ocelli  and  optic  flow  computation. 

3.2  Ocellar  Sensor 

The  ocellar  sensor  (based  on  work  by  Gremillion  et  al.42)  gives  roll  and  pitch  rate 
estimates  using  the  luminance  change  sensed  by  right-left  or  front-back 
photodiode  pairs.  The  luminance  signals  from  left  and  right  photodiodes  are  band¬ 
pass  filtered.  The  high-pass  filter  portion  serves  as  the  differentiator  element  to 
estimate  rate  information  introduced  by  luminance  change.  The  high-frequency 
cutoff  was  added  to  reject  the  flickering  noise  for  indoor  usage.  The  filtered  signals 
from  the  photodiodes  are  antagonistically  subtracted  from  each  other  (left-right  or 
front-back)  to  obtain  roll  and  pitch  rate  estimates.  The  overall  circuit  schematics 
are  shown  in  Fig.  5  and  the  circuit  components  are  listed  in  Table  1. 
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C2  470n 


Fig.  5  Circuit  schematics  of  the  3-stage  ocellar  sensor.  TSL14S  photodiode  outputs  are 
band-pass  filtered  and  antagonistically  subtracted.  Pitch  rate  (front-back)  is  inverted  for  sign 
change. 


Approved  for  public  release;  distribution  is  unlimited. 

8 


Table  1  Circuit  components 


Component 

Value/part  number 

Rl,3,9,ll 

1.1  kO 

R5,6,7,8,13,14,15,16,17,18,19,20 

1  kO 

R2,4,10,12 

20  kO 

Cl.3,6,8 

1  pF 

C2,4,7,9 

470  nF 

Operational  amplifier 

ISL28208 

photodiode 

TSL14S 

Vdd 

5  V 

The  circuit  consists  of  3  stages: 

1)  Light-to-Voltage  Conversion 

Light-to-voltage  conversion  by  a  TSL14S  optical  sensor43  that  combines  a 
photodiode  and  a  transimpedance  amplifier.  The  sensor  has  a  wideband  spectral 
response  characteristics  between  320  and  1050  nm.  Its  peak  output  is  at  640  nm. 
The  output  voltage  from  this  element  is  the  electrical  equivalent  of  luminance  seen 
by  the  photodiode. 

2)  Band-Pass  Filtering 

This  stage  consists  of  an  active  bandpass  filter  with  a  designed  high-pass  cutoff  at 
17  Hz  and  low-pass  cutoff  at  145  Hz  (see  Fig.  6). 


Cf 


Fig.  6  Band-pass  filter,  with  high-pass  cutoff  at  17  Hz  and  low-pass  cutoff  at  145  Hz 

The  input-output  relationship  of  a  high-pass  filter  is  modeled  as 


V0(t)  =  -RfCsji(VtV))  .  (1) 
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The  input  voltage  is  the  luminance  value  from  the  TSL14S  package.  The  output 
voltage  approximates  the  luminance  time  rate  of  change.  The  function  of  the  low- 
pass  filter  is  to  attenuate  high-frequency  noise.  The  final  band-pass  filter  transfer 
function  is  in  Eq.  2. 

To (s)  _ _ sCsRp _ 

V;(s)  s2CpRf  CsRs+s(CpRp +C$Rs)+l 


The  characteristic  quantities  of  this  second  order  transfer  function  are  the  low-pass 
cutoff  frequency  mL,  high-pass  cutoff  frequency  ojh,  and  maximum  input-output 
gain  Amax,  specified  in  Table  2. 

Table  2  Band-pass  filter  characteristics 

1  106  rad/s  (16.9  Hz) 

<X>L  ~  RFCF 

1  909  rad/s  (145  Hz) 

-R^s 

„  -Rf  -18.2 

A  —  _ 

™ max  n 

3)  Linear  combination  stage: 

This  stage  includes  a  difference  amplifier  to  subtract  right-left  filter  outputs  and 
front-back  filter  outputs.  The  difference  amplifier  output  from  the  right-left  inputs 
estimates  the  roll  rate.  The  difference  amplifier  output  from  the  front-back  inputs 
is  inverted  (for  sign  change)  by  an  inverting  amplifier.  Inverting  amplifier  output 
estimates  the  pitch  rate  (see  the  blocks  in  Fig.  7). 


Fig.  7  Subtractor  and  inverter:  Subtractor  is  used  for  antagonistic  subtraction  of  filtered 
signals.  Inverter  is  used  for  sign  change  for  pitch  rate.  For  equal  resistors  in  both  blocks,  direct 
subtraction  and  direct  inversion  is  satisfied. 


For  Ri, 23,4,5,6  =  1  kfk  the  outputs  V0_diff  and  V0_inv  are  modeled  as 


Vo  Jiff  =  Vi  2-V 


il 


Vn 


=  -Vi 


(3) 
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(4) 


The  bandpass  filter  was  simulated  (Fig.  8)  using  the  macro  model  of  the  ISL28208 
operational  amplifier  in  the  Tina  TI  SPICE-based  simulation  program.  The 
simulated  circuit  and  alternating  current  (AC)  transfer  characteristics  for 
frequencies  between  1  mHz  and  1  MHz  are  seen  in  Fig.  9.  According  to  the 
simulation  (due  to  the  zero  in  the  denominator  of  the  transfer  function)  the 
amplitude  is  increased  by  20  dB/decade  until  it  hits  the  first  pole.  The  high-pass 
-3  dB  frequency  is  seen  as  13  Hz.  The  maximum  amplitude  is  5.58  dB  around  55 
Hz.  The  low-pass  -3  dB  frequency  is  seen  as  175  Hz. 


r."  .ret! 


Fig.  8  Simulated  circuit  in  TI  TINA  simulation  software 


Fig.  9  Band-pass  filter  simulation  results:  13.78  Hz  high-pass  cutoff  and  174  Hz  low-pass 
cutoff  is  observed.  Phase  starts  at  -90°  at  1  mHz  and  reaches  to  -270°  at  100  kHz. 


Approved  for  public  release;  distribution  is  unlimited. 

11 


3.3  Mathematical  Model  for  the  Ocellar  Sensor 


With  reference  to  Fig.  10,  the  variables  used  to  explain  the  ocellar  sensor  are  as 
follows: 

y:  Azimuth  angle  of  the  photodiode 
cp :  Angular  position 
0:  Photodiode  field  of  view 
d  ~  ^  ~  Y:  Angular  speed 

L(y):  Luminance,  assume  periodic 
a:  Light  source  field  of  luminance 

r  =  o 

(p,  angular  position 

=  n/2 


Arbitrary  luminance 
distribution 

Photodiode  in  rotational  motion 


Luminance  perceived  by 
photodiode 


1 

-0/2 

J _ 

0/2 

0  y •  azimuth  angle 


Luminance  sourced  by  DC 
spotlight 


-o/2 

1 

0/2 

0  Y’  azimuth  angle 


Fig.  10  Mathematical  model  and  assumptions:  Photodiode  in  rotational  motion  sees  the 
arbitrary  luminance  pattern  as  its  azimuthal  angle  varies 

The  test  setup  has  a  DC  spotlight  source,  which  acts  as  “the  sun”.  For  simplicity, 
luminance  is  modeled  as  a  rectangular  function  with  fixed  edges  from  -  a/2  to  a/2. 
Photodiode  FOV  is  also  modeled  as  a  rectangular  function  with  edges  at  FOV 
edges,  —0/2  to  0/2 

h(0,  y):  Photodiode  field  of  view  function  modeled  as  a  rectangular  filter  with 
edges  at  —0/2  and  0/2 

p :  Photodiode  output  (or  signal  input  to  bandpass  filter) 
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*:  convolution  operation 

The  photodiode  integrates  the  luminance  in  its  FOV,  as  h(0,y) 

P  =  fF0V ,L(y)dY  =  Uy)  *  K0,y)  =  Lh(y )  ■  (5) 

Let  Lh(y)  be  a  photodiode  output  taken  at  the  azimuth  angle  y.  While  the  circuit  is 
in  rotational  motion  with  angular  speed  1 9,  this  photodiode  output  becomes 
Lh(y  ~  At)  at  time  t.  Thus,  the  photodiode  output  encodes  both  spatial  (7)  and 
temporal  (dt)  information. 

Ln(r)  Lh(y--dt)  ■  (6) 

Since  the  photodiode  output  is  in  2  domains,  the  Fourier  transform  with  respect  to 
both  spatial  and  temporal  variables  should  be  taken  in  order  to  express  it  in  Fourier 
domain. 

Lh(y,  t):  denotes  the  function  in  spatial  and  temporal  domain 
Lhf  fy,  ft)'-  denotes  the  Fourier  transform  Lh(y,  t ) 

FTy  ,t 

Lh(y  -  tit)  Lh(fy,  ft)  ■  (7) 

Properties  used: 

Shifting  property  in  time/space  and  Fourier  domain: 

FT 

x(y  -  /?)  <4 X(fY)e-i2nW  .  (8) 

Convolution: 

FT 

x(y)*y(y)<^x(fy)Y(fY)  .  (9) 

Taking  the  Fourier  transform  with  respect  to  spatial  variable  y: 

FT 

.  (10) 

Then,  taking  the  Fourier  transform  with  respect  to  temporal  variable  t  gives  the 
frequency  domain  of  a  rotational  motion: 

P{fy,ft)  =  Vh(jy)S{ft  +  dfy)  (11) 

FT 

Lh(y-m)^rh(fY)5(ft  +  dfY)  .  (12) 
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This  means  that  all  the  energy  of  the  rotating  photodiode  output  is  contained  in  a 
plane  of  the  spatiotemporal  frequencies  domain.44-45  The  equation  of  this  plane  is 

ft+0fY  =  0  •  (13) 

The  rectangular  luminance  function  can  be  described  as  Lh(y  —  tit)~rect(x,t) 
with  space  and  time  axis.  A  rectangular  pulse  in  space  rest(x)  has  sine  form  in 
frequency  domain: 


rect(x)g!i!25E«  .  (14) 

nfx 

The  corresponding  frequency  spectrum  appears  as  a  cut  of  spatial  spectrum46 

sm(  ~^x)  by  a  wall  of  Dirac  situated  in  the  direction  of  ft  +  tifv  —  0 

nfx  Jy 

Overall, 


LhUr’ft)  =  Lh(fy)S(ft  +  tifY)  .  (15) 

Equation  15  represents  the  photodiode  output  given  as  input  to  the  bandpass  filter 
in  the  circuit.  Bandpass  filtering  is  a  temporal  process;  thus,  the  bandpass  function 
has  only  temporal  variable.  Let  the  bandpass  filter  transfer  function  in  time  and 
Fourier  domain  be  defined  as 


FTt 


BPF(t)  <=>  BPF(ft)  orBPF(s) 
From  the  ocellar  sensor  section,  BPF(s)  is  defined  as 


BPF(s )  = 


v0(s) 


sCsRp 


(/[(s)  S2CfRfCsRs+s(CfRf+CsRs)+1 
The  output  of  the  bandpass  filter  can  be  written  as 

W ,bpf(Y>  t)  =  Lh(y  —  tit)  *  BPF(t )  . 


The  Fourier  transform  of  the  output  becomes 


Jh,BPF 


(y,  0 


FT, 


Y,t 


Jh,BPF 


C fvft ) 


FT, 


Lh(y  -  tit)  *  BPF(t)  ^  Lh(fY)8(ft  +  tifY)BPF(ft) 


(16) 

(17) 

(18) 

(19) 

(20) 


This  output  represents  the  luminance  output  filtered  by  one  photodiode.  Assume 
another  photodiode  is  has  a  different  azimuth  angle,  y  —  /?,  offset  by  /?  from  the 
first  photodiode.  The  luminance  perceived  by  it  will  be 
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(21) 


F7V,t  —  , 

Lh,p  (.Y  ~~  P  ~  fit)  ' - *  Lh,p  ( fy ,  ft ,  /?) 

FTv,t _ .  , 

Lh,p,BPF(y>t, P)  < — >  Lh,/3,BPF  {fy.ft.P)  (22) 

Q(/,./J)  =  Cf(/,)«tt  +  e/,)f‘J'2"w  ■  (23) 

After  bandpass  filtering,  the  Fourier  transform  of  the  second  output  becomes 

L^pfOyJvP)  =  W(/y)5(A  +  Vfy)e-J2nWBPF(ft)  .  (24) 


The  difference  amplifier  implements  direct  subtraction  between  2  bandpass  filter 
outputs  as  the  roll  rate  estimation.  Let  the  roll  rate  be  denoted  as  p(y,t) 

PO.  t)  =  Lh,bpf(Y ,  0  -  L h,p,BPF(y>  0  (25) 

FT 

p(y,t)<^>P  (26) 

p ifyJt)  =  TPJpF(fy.ft)  -  LwpF(fy’ftjO  =  rh(fy)5(ft  +  dfy)BPF (A)  - 
LhAfy)5(ft  +  VfY)e-j2nWBPF{ft)  =  BPF(Jt)S{ft  +  dfy)[Lh{fy )  - 

W(/y)e-;'27r/y/?]  •  (27) 

Thus,  the  circuit  output  depends  on 

•  Photodiode  FOV 

.  Bandpass  filter  characteristics 

•  Luminance  function  L  ( y ) 

•  Photodiode  angular  separation  /? 

The  photodiode  field-of-view  and  bandpass  filter  characteristics  are  inherent  in  the 
circuit  under  test,  and  these  variables  are  fixed.  We  have  control  of  the  luminance 
function  L(y )  and  photodiode  angular  separation  /?.  The  luminance  function  is  also 
dependent  on  the  light  intensity  (or,  the  input  power  given  to  light  source). 

3.4  Optic  Flow  Computation 

Optic  flow  is  an  approximation  of  apparent  motion  of  brightness  patterns  observed 
when  an  observer  (i.e.,  camera)  is  moving  relative  to  the  objects  it  images.  Optic 
flow  methods  try  to  calculate  where  a  pixel  in  Image  A  goes  to  in  a  consecutive 
Image  B.  In  2  dimensions,  optic  flow  specifies  how  much  a  pixel  of  an  image  moves 
between  adjacent  series.47  The  basis  of  optic  flow  is  the  brightness  constancy 
equation,  which  eventually  forms  the  2-D  motion  constraint. 
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Assume  that  I(x,y,t)  is  the  intensity  of  pixel  positioned  at  location  (x,y)  in  a  frame 
taken  at  time  t.  In  the  frame  taken  at  time  (t+At),  this  pixel  moves  to  the  location 
(x+Ax,  y+Ay)  (see  Fig.  1 1). 


Frame  (t)  Frame  (t+At) 


(x,y,t)  (x+  Ax;y+Ay;t+At) 


Fig.  11  Optic  flow  vector  for  a  pixel  between  2  consecutive  frames 
Assuming  the  brightness  of  the  pixel  does  not  change  over  time; 

/(x,y,  t)  =  /(x  +  Ax,y  +  Ay,  t  +  At)  .  (28) 

Performing  first-order  Taylor  Series  expansion  about  I  (x,  y,  t): 

/(x  +  Ax,y  +  Av,  t  +  At)  =  /(x,y,  t)  +  Ax  +  Ay  -I-  At  + 

ox  dy  dt 

higher  order  terms  .  (29) 

Assuming  very  small  motion  and  ignoring  the  higher-order  terms, 

J-Ax  +  ^-Ay  +  J-At  =  0  .  (30) 

dx  dy  *  dt 


Dividing  everything  by  At: 


Denoting: 


dl  Ax  dl  Ay  d I  At 
dx  At  dy  At  dt  At 


(31) 


91  —  j  91  —  i  91  — 

dx  x’  dy  y  dt  t 


-  = 


At 


At 


IxVx  +  lyVy  +  4  —  0 


(32) 


(33) 


Here,  14  and  Vy  are  the  x  and  y  components  of  optic  flow  (for  the  motion  described 
in  Fig.  14).  The  equation  can  be  written  more  compactly  as 
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(4-4) '  64- ^y)  =  4 

(34) 

V/  ■  <9  =  -4 

(35) 

where  V/  =  (lx,  /y)  is  the  spatial  intensity  gradient  and  d  =  (dx,  i9y)  is  the  velocity 
of  the  pixel  (x,y)  at  time  t.  Equation  35  is  called  the  2-D  motion  constraint  equation. 
This  equation  has  2  unknowns  (dx,dy),  which  relates  to  the  aperture  problem.  If 
the  motion  detector’s  aperture  is  much  smaller  than  the  contour  it  observes,  it  can 
only  be  sensitive  to  the  component  of  the  contour’ s  motion  that  is  perpendicular  to 
the  edge  of  the  contour.  It  is  blind  to  any  motion  parallel  to  the  contour.  This  is 
because  the  movement  in  this  direction  will  not  change  the  appearance  of  anything 
within  the  aperture.  To  find  optic  flow  vectors,  additional  equations  are  needed. 
Many  optic  flow  computation  methods  focus  on  additional  constraints  that  attempt 
to  recover  the  optic  flow  vectors.  Lucas  and  Kanade48  assume  that  the  displacement 
of  the  image  contents  between  2  frames  is  constant  within  a  neighborhood  of  a  point 
under  consideration.  Horn  and  Schunck49  assume  smoothness  in  the  flow  over  the 
whole  image  and  prefers  solutions  that  show  more  smoothness. 

For  the  Lucas-Kanade  motion  algorithm,48  the  2-D  motion  constraint  equation  is 
assumed  to  hold  for  all  pixels  within  a  window  centered  at  p.  This  means  that  the 
motion  constraint  equation  holds  for  all  the  pixels  in  a  window  with  the  same 
unknowns  d  =  (dx,  4y).  This  set  of  equations  brings  an  overdetermined  system 
that  has  more  equations  than  unknowns: 

4  (<7l)  fix  “t"  4  (<7l)  <4  —  — If  (<7l) 

4  (<72 )  +  4  (<7z)  tfy  =  ~4  (<7 2) 


4  (<7 n)  <4  4  (<7n)  *4  4  (<7 n) 

where  qq,  q2, . qn  are  the  pixels  inside  the  window.  In  matrix  form: 

Ad  =  b,  where 


A  = 


4  (<7i)  4  4i) 
4  (<72)  4  (to) 

1-4  (<7n)  4  (<7n)J 

i9v 


>9. 


7)  = 


i9  = 

I  iv, 

~k  (<7i) 

— 4  C<?2) 


Least  squares  principle  can  be  applied  to  solve  this  overdetermined  system: 

21r21i9  =  21Th 
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(36) 


0  =  (. ATA')~1ATb  (37) 


TV 

Ei4(<7i)2  ifliVy  (JhH 

-1 

4  (<7i)4  Oji) 

j9y 

Zi*y  (<7i)407i)  Y.ilyiqd2 

.  4  (<7i)4  0?i). 

Optic  flow  vectors  dx  and  dy  are  searched  in  a  tracking  window,  and  the  best  match 
is  found  using  the  least  squares  method.  This  system  is  solvable  if  A7  A  is  invertible. 
The  eigenvalues  of  A7 A  (A1,A2  where  A1  >  A2  )  should  not  be  too  small,  and  A7 A 
should  be  well-conditioned  (-1  should  not  be  too  large),  so  A1  >  A2  should  be 

somewhat  similar  to  each  other  in  magnitude.  In  other  words,  very  small 
eigenvalues  are  interpreted  as  “flat  surfaces”,  and  eigenvalues  A1  »  A2  or  A1  «  A2 
are  interpreted  as  “edges”.  Optimum  eigenvalues  should  be  large  enough  and  have 
similar  amplitude.50 

One  drawback  of  the  Lucas-Kanade  algorithm  is  that  it  theoretically  fails  for  large 
motions.  If  the  motion  is  too  large,  higher  order  terms  may  dominate  Eq.  29  (the 
first-order  Taylor  Series  Expansion).  Reducing  the  image  resolution  may  solve  this 
issue.  A  pyramidal  approach  is  available  to  convert  large  motions  to  small 
motions.51 

Different  optic  flow  computation  methods  can  be  described  as  either  “dense”  or 
“sparse.”  From  a  performance  point  of  view,  dense  computation  methods  (e.g., 
Horn  and  Schunck49  and  Farneback52)  that  process  all  of  the  pixels  in  the  image  are 
slow  for  real-time  applications.  Instead,  sparse  techniques  (i.e.,  Lucas  and 
Kanade48)  only  process  the  pixels  of  interest.  For  real-time  applications  that  use 
optic  flow  computation  to  feed  the  current  state  of  an  object  back  to  a  control  loop, 
sparse  techniques  may  be  preferred  over  dense  techniques  due  to  faster 
computational  performance  (and,  thus,  higher  sampling  rate).  In  practice,  we 
achieved  60  frames  per  second  (fps)  using  the  Lucas-Kanade  algorithm  but  only 
13  fps  for  the  Farneback  algorithm  (376  x  240  pixels  8-bit  monochromatic  image 
sequence).  For  this  work,  the  Lucas-Kanade  algorithm  is  used  with  predefined 
feature  points  distributed  over  the  imagery.  The  feature  points  are  the  center  pixel 
points  to  run  the  Lucas-Kanade  algorithm  for  determining  an  optic  flow  vector.  As 
the  number  of  feature  points  increases,  so  does  the  number  of  optic  flow  vectors. 
The  x-component  of  optic  flow  vectors  are  summed  to  obtain  a  single  optic  flow 
value. 


3.5  Experimental  Setup 

In  general,  from  Fig.  12,  the  optic  flow  experienced  by  an  imager  is33 

Optic  Flow  —  —co  +  ( V/D )  cosQ 
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(39) 


where  oo  is  the  angular  and  V  is  the  translational  velocity  of  the  vehicle,  D  is  the 
distance  to  an  object,  and  9  is  the  angle  between  direction  of  travel  and  direction  of 
object.  If  the  translational  component  (V)  is  zero,  optic  flow  is  proportional  to  the 
angular  velocity. 


Imager 


Fig.  12  Optic  flow  during  rotational  and  translational  motion:  without  translational 
component  (V),  optic  flow  is  an  estimate  of  only  angular  velocity  (to) 


A  mechanism  was  constructed  to  characterize  optic  flow  and  ocellar  sensor  over 
0.1-10  Hz  rotational  mechanical  input.  Figure  13  shows  the  illustration  of  the  test 
setup;  Fig.  14  shows  the  scene  the  camera  sees;  Table  3  shows  the  system 
components;  and  Figs.  15  and  16  show  the  general  and  close-up  views  of  the 
components.  The  block  diagram  of  the  system  is  shown  in  Fig.  17. 


MCU 


gyro 


ADC  /  ocelli 


USB  hub 

Camera  +  fisheye  lens 
Servo  motor 

Aluminum  platform 


DC 


DC  light  source 


Light  Source 
Supply 


Fig.  13  Illustration  of  test  setup:  Light  source  has  its  own  DC  supply  to  avoid  issues  of 
flickering;  information  from  camera,  microcontroller  unit  (MCU),  gyro,  and  analog-to-digital 
converter  (ADC)  are  transferred  to  the  host  computer  via  a  USB  hub. 
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Fig.  14  Camera  scene  (376  x  240  pixel  image):  DC  light  source  is  not  in  the  FOV  of  the 
camera,  which  is  moving  along  the  x-direction 


Table  3  Experiment  components 


Equipment 

Model/manufacturer 

Light  source 

LED1  OOWA-56  LED  Video  light 

Light-source 

supply 

GW-Instek-PSW-8027  Programmable  switching  DC  power  supply 

Motor 

Animatics  Smartmotor  SM2340D 

Motor  supply 

PS42V6AG-1 10,  251  W,  Moog,  Animatics 

Signal  generator 

Tektronix  AFG3252 

Camera 

UEYE  UI-1221LE-M-GL  USB  2.0.752  x  480.  CMOS  mono,  87.2  fps,  8- 
bit 

Sunex  DSL227  Miniature  superfisheye  lens,  180°  FOV 

Lens 

Microcontroller 

Arduino  UNO 

ADC 

MCP3008  8 -channel  10-bit  ADC  with  serial  peripheral  interface  (SPI) 

USB  hub 

Hosa  Technology 

Gyroscope 

Pololu  MinIMU-9  v3  chip  contains  L3GD20H  3-axis  gyro 
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DC  Light 
Source 


Vicon 

markers 


Motor,  camera,  gyro, 
ADC,  MCU 


Fig.  15  Overall  test  setup:  Ocellar  sensor  is  positioned  in  front  of  light  source.  Motor  is 
giving  rotational  motion  to  the  setup  along  its  shaft  axis.  The  motor  shaft  is  in  vertical 
orientation,  moving  the  components  on  it. 


Fig.  16  Test  setup  components:  Camera  sees  the  scene  shown  in  Fig.  17.  Camera  on  the  right 
is  not  used  due  to  performance  issues. 
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Through 
USB  hub 


USB,  60  fps 
376x240  px 


r-— » 

ROS  w/  chrony 


Camera  + 
fisheye  lens 


Vicon 

Motion 

Detection 

System 


Animatics 

Servo 

Motor 


Motor  Supply 


_ i 


For  the  future 


Fig.  17  System  block  diagram:  All  of  the  data  collected  are  stored  in  laptop 

All  data  are  stored  in  a  laptop  running  Ubuntu  14.04  environment.  The  ROS 
environment  is  used  to  implement  robot  software.  ROS  is  an  open-source  network 
for  writing  robot  software,  including  a  collection  of  tools,  libraries,  and 
conventions.  It  allows  for  compact  storage  and  data  publishing  from  multiple 
peripherals.  For  this  system,  each  component  is  represented  by  a  different  ROS 
“node”  that  allows  for  the  compilation  of  multiple  C++  files  and  stores  data  in  a 
“bag”  file.  Once  the  data  are  collected,  the  bag  file  is  “unbagged”  and  parsed  to 
extract  the  data. 

To  record  the  ocellar  sensor  analog  voltage  outputs,  an  ADC  board  is  used.  For 
ground  truth,  a  16-bit  gyroscope  is  also  used.  The  ADC  board  (MCP3008) 
communicates  with  the  Arduino  UNO  microcontroller  via  SPI.  Gyroscope 
communication  is  through  interintegrated  circuit  technology  (I2C).  The 
microcontroller  is  programmed  in  C  language.  It  reads  ADC  and  gyroscope  outputs, 
and  parses  the  values  into  most  significant  (MSB)  and  least  significant  (LSB)  bytes. 
Each  sample  consists  of  one  MSB  and  one  LSB  (i.e.,  a  2-byte  word).  A  message  is 
created  (see  Fig.  18)  with  2  header  bytes  to  be  sent  to  the  laptop  via  serial 
communication  at  115200  bps.  A  15-ms  delay  is  added  between  each  byte  sent  to 
allow  the  receiver  buffer  to  be  cleared  to  avoid  overwriting. 

Header  Data 

I  _ _ I _ 

1 1  ' 

|  0x63  |  0x96  |  MSB1  |  LSB!  |  MSB2  |  LSB2  |  MSB3  |  LSB3  |  MSB4  |  LSB4  |  MSB5  |  LSB5  |  MS06  |  LSB6  |  MSB7  |  LSB7  |  MSB8  |  LSB8 

Fig.  18  Serial  message  structure  from  ocelli  to  microcontroller  includes  2  header,  ocelli  data, 
and  gyro  data  bytes 


Approved  for  public  release;  distribution  is  unlimited. 

22 


The  servo  motor  can  be  operated  in  either  position  or  velocity  mode.  Velocity  mode 
does  not  offer  control  in  position.  The  motor  is  controlled  by  sending  serial 
messages  in  Ani-Basic  language.  (The  command  information  and  serial 
communication  are  specified  in  the  developer’s  guide.53)  For  both  position  and 
velocity  modes,  specific  trajectory  files  are  created  in  MATLAB,  including 
velocity/position  trajectory  (e.g.,  sine  wave,  square  wave)  and  acceleration 
information.  These  trajectories  are  recorded  as  text  files  and  read  by  a  C++  code 
that  communicates  with  the  motor  in  Ani-Basic  Language  via  serial  communication 
at  115,200  bps.  The  commanded  position,  velocity,  acceleration  and  real-time 
position,  velocity,  and  acceleration  values  can  be  read  back  from  the  motor 
controller.  Reading  the  motor  values  allowed  us  to  validate  the  gyroscope  output 
and  to  see  whether  the  ocelli  circuit  is  in  desired  position  or  not.  To  measure 
frequency  response  characteristics,  concatenated  sine  waves  are  sent  as  velocity 
trajectory  at  different  frequencies  at  velocity  mode.  To  understand  the  ocellar 
sensor  validity  (explained  in  the  next  section),  step  inputs  are  sent  at  position  mode. 

After  storing  the  gyro,  ocelli,  Vicon,  and  raw  imagery  data  in  a  bag  file,  post¬ 
processing  occurs  (and  is  summarized  in  Fig.  19).  Using  the  raw  imagery  in  the  bag 
file,  optic  flow  field  vectors  are  computed  with  another  C++  code,  using  the 
OpenCV  Lucas-Kanade  algorithm.  A  text  file  is  generated  that  includes  the  time 
stamps  and  optic  flow  x  and  y  vectors  for  each  image.  For  the  other  data,  the  bag 
file  is  unbagged  and  parsed  to  extract  the  ocelli,  gyro,  Vicon,  motor  data,  and 
related  timestamps.  Optic  flow  vectors  generated  from  the  Lucas-Kanade 
algorithm  are  also  parsed.  The  text  file  contains  60-fps  optic  flow  information.  To 
obtain  lower  frame  rate  results,  the  optic  flow  values  are  downsampled  to  factors 
of  60.  Since  we  have  data  coming  from  different  sources  at  different  sampling  rates, 
synchronization  is  necessary  using  a  common  time  vector.  A  common  time  vector 
for  all  data  is  created  with  the  lowest  sampling  time  possible,  which  is  the 
microcontroller  sampling  time,  0.003  s.  All  data  are  interpolated  using  this  time 
vector. 

Since  the  communication  modules  are  developed  for  embedded  platforms,  it  will 
not  take  much  effort  to  transfer  this  system  to  onboard  computers  used  on  quadrotor 
helicopters  (quadcopters)  through  Secure  Shell  (SSH).  The  time  synchronization 
can  be  carried  out  with  firmware  that  force  ones  computer’s  clock  to  follow  another 
one  (e.g.,  “chrony”  synchronization  for  ROS). 
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MATLAB  (Ubuntu,  Windows) 


1 

1 

Unbag  and  Parse  ocelli,  gyro,  OF, 

Interpolate  to  highest 
rate  possible  (MCU, 
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Process  for  System 

Ubuntu,  ROS 

motor,  Vicon  data  &  timestamps 
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1 

Bag  File 
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Fig.  19  Post-processing  block  diagram:  Optic  flow  vectors  are  computed  and  extracted  as  a 
text  file.  The  bag  file  is  parsed,  interpolated,  and  processed  for  data  analysis. 


3.6  Magnitude-Squared  Coherence 


The  spectral  coherence  is  a  measure  that  can  be  used  to  examine  the  relationship 
between  2  signals  (or  data  sets).  It  is  commonly  used  to  estimate  the  power  transfer 
between  the  input  and  output  of  a  linear  system.  The  magnitude- squared  coherence 
between  two  signals  x(t)  and  y(t)  is  defined  as 


c -(f)  =  |G*y(/)|2 

XyK1  J  GXX(f)Gyy(f ) 


(39) 


where  Gxy(f )  is  the  cross-spectral  density  between  x  and  y,  and  Gxx  (/),  Gyy  (/)  are 
the  auto-spectral  density  of  x  and  y,  respectively.  If  the  signals  are  ergodic 
(statistical  properties  can  be  deduced  from  a  sufficiently  long  process)  and  the 
system  function  linear,  the  magnitude- squared  coherence  function  estimates  the 
extent  to  which  y(t)  may  be  predicted  from  x(t)  by  an  optimum  linear  least  squares 
function.54  The  transfer  functions  and  operations  described  for  the  mathematical 
model  of  the  system,  itself,  and  ocellar  sensor  transfer  characteristics  are  linear. 
Thus,  we  expect  the  system  to  be  linear.  The  magnitude- squared  coherence  is  added 
to  the  frequency  response  plots  as  a  performance  parameter  showing  linearity. 

For  an  ideally  linear  system: 

y(t)  =  x(t)  *  h(t)  Y (/)  =  X(f)H(f ) 

GyyC/0  =  m)\2Gxx{f) 

GXy(f )  =  m)\2Gxx{f) 

c  _  \H(f)Gxx<j)\2  _  \H(f)Gxx{f)\2  _  1 

XyVI)  Gxx(f)Gyy(f)  Gxx(f)2\H(f)\2 


(40) 

(41) 

(42) 

(43) 
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where  h(t)  is  the  impulse  response  and  H  (/)  is  its  Fourier  transform. 

Values  of  coherence  satisfy  0  <  Cxy  (/)  <  1.  If  there  is  a  perfect  linear  relationship 
between  x  and  y  at  a  given  frequency,  Cxy  (/)  =  1.  If  Cxy  is  less  than  one  but  greater 
than  zero,  it  is  an  indication  that  either  noise  is  an  inherent  component  of  the  system 
measurement,  that  the  assumed  function  relating  x(t)  and  y(t)  is  not  linear,  or  that 
y(t)  is  producing  output  due  to  input  x(t)  as  well  as  other  inputs.  If  the  coherence  is 
equal  to  zero,  it  is  an  indication  that  x(t)  and  y(t)  are  completely  unrelated. 

In  the  physical  world,  a  perfect  linear  relationship  is  rarely  realized.  In  practice, 
coherence  values  higher  than  0.5  are  acceptable  for  testing  linear  systems.  For  the 
experiments  described  later,  the  coherence  values  dip  down  at  specific  frequencies, 
around  1-2  Hz,  for  all  measurements.  Although  the  input  sine  waves  includes  these 
frequencies,  it  is  believed  that  the  motor  was  not  successful  at  implementing  these 
frequencies.  All  of  the  final  measurements  include  coherence  values  higher  than 
0.5  to  be  in  a  practically  acceptable  region. 

3.7  Ground  Truth 

The  first  consideration  for  ground  truth  was  the  Vicon  motion-detection  system. 
However,  its  cameras  strobe  at  frequencies  50-100  Hz  and  use  reflected  infrared 
light  that  is  strobed  from  a  ring  of  light-emitting  diodes  (LEDs)  surrounding  each 
camera.  The  bandpass  filter  circuit  is  able  to  pick  up  these  frequency  components 
of  the  infrared  light,  resulting  in  the  corruption  of  the  output  signal.  Alternative 
ground-truth  options  are  the  velocity  readout  from  the  servo  motor  controller  and 
the  gyroscope  sensor.  To  choose  one  as  ground  truth,  a  chirp  signal  between  0.1 
and  10  Hz  was  given  to  the  motor  as  velocity  input,  and  the  comparative  responses 
of  motor  velocity  readout  and  gyroscope  were  verified,  with  respect  to  the  Vicon 
system  as  input.  The  ideal  response  should  be  a  flat  curve.  As  seen  in  Fig.  20,  the 
gyroscope  provides  a  more  flat  magnitude  very  close  to  0  dB,  and  less  phase  delay 
than  motor  velocity.  The  magnitude- squared  coherence  plot  indicates  a  linear 
input-output  relationship  at  an  existing  frequency.  The  gyroscope  coherence  is 
higher  than  motor  velocity  across  all  test  frequencies;  therefore,  it  is  regarded  as 
ground  truth  for  further  analysis.  The  curves  are  shown  up  to  15  Hz  to  show  the 
coherence  decay  outside  the  test  frequencies. 
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Fig.  20  Motor  velocity  and  gyro  frequency  response,  as  seen  by  Vicon  motion-detection 
system  as  input:  Frequencies  after  10  Hz  were  shown  to  prove  the  decrease  in  coherence  out 
of  controlled  motion  frequencies.  Gyroscope  shows  a  flatter  magnitude  response  and  higher 
coherence  than  motor  velocity;  therefore,  it  was  chosen  to  be  the  ground  truth. 


3.8  Understanding  Ocellar  Sensor's  "Valid  Range" 

The  ocelli  circuit  is  assumed  to  work  under  a  specific  luminance  pattern  to  be  an 
angular  rate  sensor.  Assuming  a  bright  sky  and  dark  ground,  when  the  photodiodes 
are  looking  to  the  sides,  each  of  them  sees  a  different  horizon.  One  photodiode  sees 
a  brighter  patch,  while  the  other  one  sees  a  darker  patch.  For  this  algorithm  to  work, 
the  luminance  gradient  from  the  sky  to  the  ground  should  be  constant  and  negative. 
We  use  a  light  source  with  a  diffuser  to  create  this  artificial  sky  and  horizons.  The 
diffuser  prevents  the  direct  current  (DC)  source  from  acting  as  a  point  source,  by 
helping  to  distribute  the  light  intensity  along  the  diffuser  surface.  This  way,  the 
light  source  acts  as  the  sky,  rather  than  the  Sun.  The  photodiodes  should  see  the  2 
edges  of  the  source  as  2  horizons.  This  way,  when  a  rotational  motion  is  applied, 
brighter-darker  patch  assumption  will  be  satisfied.  The  photodiodes  should  be  bent 
towards  the  source  to  intersect  their  fields  of  view.  To  understand  whether  an 
intersection  is  created,  the  azimuthal  position  of  the  motor  is  varied  and  the 
photodiode  outputs  are  checked  to  see  if  they  share  an  overlapping  FOV.  Figures 
21  and  22  show  the  unbent  and  bent  raw  photodiode  outputs,  with  respect  to  the 
azimuthal  position  of  the  motor.  A  partially  overlapping  FOV  was  achieved  by 
bending  the  photodiodes  toward  the  light  source. 
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Motor  shaft  azimuthal  position  (rad) 


Fig.  21  Unbent  photodiode  output  vs.  motor  shaft  azimuthal  position:  Photodiode  outputs 
increase  as  they  pass  by  the  light  source.  FOVs  are  not  overlapping. 


Motor  shaft  azimuthal  position  (rad) 


Fig.  22  Bent  photodiode  output  vs.  motor  shaft  azimuthal  position:  Photodiode  FOVs  are 
partially  overlapping,  which  is  required  for  the  ocellar  sensor  to  work.  In  this  (incorrect) 
configuration,  there  are  angles  where  simulated  roll  motions  do  not  produce  any  change  in  the 
photodiode  outputs. 
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Due  to  the  small  size  of  the  light  source  and  small  FOV  of  the  photodiodes  (around 
90°  each),  the  maximum  swing  of  the  motion  stimulus  needs  to  be  small  enough  to 
ensure  that  the  photodiode  outputs  are  changing  symmetrically  with  respect  to  each 
other.  To  understand  the  dynamic  range  circuit’s  velocity,  the  azimuth  was  varied 
with  small  steps.  For  each  position  value,  a  target  velocity  was  given  to  the  motor 
and  the  circuit  output  was  compared  to  the  gyro  output.  For  the  region  outside  the 
light  source  dominance,  the  circuit  outputs  are  not  reliable. 

Figure  23  shows  a  valid  region  (azimuthal  position  changes  from  -0.2  to  0.2 
radians).  In  this  range,  photodiode  outputs  are  symmetric  to  each  other  and  the 
ocelli  output  directionally  matches  the  gyro  output. 

Figure  24  shows  an  invalid  region  (azimuthal  position  changes  from  -1.4  to  -0.2 
radians).  In  this  range,  photodiode  outputs  are  not  symmetric  to  each  other.  Ocelli 
is  not  in  agreement  with  gyro.  Using  this  data,  the  maximum  displacement  for  the 
ocellar  circuit  is  determined  to  be  1  radian.  All  of  the  following  characterizations 
are  done  in  this  valid  region. 


Time  (s) 


Time  (s) 


Fig.  23  Ocelli  in  valid  range:  (above)  symmetric  photodiode  raw  output;  (below)  gyro  and 
ocelli  output  for  motor  azimuthal  position  (-0.2  to  0.2  radians).  Ocelli  output  is  in  agreement 
with  gyro  in  this  range. 
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Fig.  24  Ocelli  in  invalid  range:  (above)  asymmetric  photodiode  raw  output;  (below)  gyro 
and  ocelli  output  for  motor  azimuthal  position  (-1.2  to  0.2  radians).  Ocelli  output  is  not  in 
agreement  with  gyro  in  this  range. 

3.9  Ocellar  Sensor  Frequency  Characterization 

Frequency  characterization  is  done  in  2  ways.  First,  to  confirm  the  proper  operation 
of  the  bandpass  circuit,  the  raw  photodiode  output  is  used  as  input  and  filtered 
signal  is  used  as  output.  Although  the  bandpass  circuits  can  operate  up  to  higher 
frequencies,  the  motor  stimulus  is  limited  to  10  FIz;  only  0.1-10  Hz  data  are 
obtained  for  circuit’s  motion  characterization.  To  demonstrate  that  the  photodiode 
and  bandpass  filter  combination  can  operate  at  higher  frequencies,  an  LED  was 
driven  by  a  signal  generator  between  3  and  100  Hz. 

A  second  frequency  characterization  was  done  to  compare  the  optic  flow  and  ocelli 
outputs  with  the  gyro  as  ground  truth.  This  provided  us  with  information  about  how 
well  the  sensors  operate  within  the  motion  frequencies,  in  comparison  to  each  other 
with  the  same  inputs. 

3.9.1  Circuit  Frequency  Characterization 

A  chirp  signal  is  given  as  a  motor  velocity  command.  Figure  25  shows  the  simulated 
circuit  output,  and  Fig.  26  shows  the  right  and  left  circuit  frequency  responses. 
Between  0.1  and  10  Hz,  the  circuit  simulation  shows  20  dB/decade  increase  in 
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magnitude,  starting  from  -38  dB  to  1  dB.  The  phase  delay  starts  from  -90°  and 
reaches  to  -125°  at  10  Hz.  The  data  from  both  photodiodes  show  the  similar 
response.  Right-circuit  magnitude  starts  from  -38  dB  and  reaches  to  -2  dB. 
Left-circuit  magnitude  starts  from  -38  dB  and  reaches  to  0  dB.  Phase  response 
reaches  to  -140°  and  -124°  for  right  and  left,  respectively,  after  starting  from  -90°. 


Fig.  25  Band-pass  filter  simulated  AC  transfer  characteristic  at  0.1-10  Hz:  Magnitude 
increases  with  20  dB/decade.  Phase  drops  from  -90°  to  -125°  at  the  end  of  10  Hz. 


Fig.  26  Right  and  left  band-pass  filter  measured  AC  transfer  characteristics  at  0.1-10  Hz: 
Magnitude  and  phase  plots  are  in  agreement  with  simulation  (Fig.  28). 
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To  demonstrate  that  the  photodiode  and  bandpass  filter  combination  can  operate  at 
higher  frequencies,  an  LED  was  taped  to  one  photodiode  (see  Fig.  27).  The 
electrical  signal  (sine  wave,  voltage-controlled)  from  the  signal  generator  is  swept 
from  3  to  100  Hz.  Figure  28  shows  the  AC  characteristics  simulation  from  3  to 
100  Hz.  The  gain  starts  from  -8.59  dB  and  reaches  to  5.6  dB  at  50  Hz,  then  it 
decreases  to  4.74  dB  at  100  Hz.  The  phase  starts  from  -101°  at  3  Hz,  decreasing  to 
-205°  at  100  Hz.  Figure  29  shows  the  frequency  response  of  the  circuit  from  the 
photodiode  input  from  FED  to  the  filtered  output.  The  magnitude  response  starts 
from  -10.25  dB  and  reaches  to  -0.29  dB  at  50  Hz.  Then  it  decays  to  -1.281  dB  at 
98.6  Hz.  The  phase  response  starts  from  -105.8°  at  3  Hz,  decreasing  to  -180.5°  at 
98.76  Hz.  Qualitatively,  circuit  simulation  is  close  to  actual  data.  Circuit  simulation 
results  in  103°  phase  delay,  and  data  result  in  75°  phase  delay.  The  circuit  gain 
increases  by  14  dB  up  to  50  Hz,  and  real  data  gain  increases  by  10  dB. 


Fig.  27  LED  sweeping:  LED  was  taped  to  photodiode  and  power-supply  signal  is  swept 
between  3  and  150  Hz. 
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Fig.  28  Right  and  left  band-pass  filter  simulated  transfer  characteristics  between  1  and  100 
Hz;  simulation  is  shown  to  compare  with  LED  sweeping  results  in  Fig.  32 
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Frequency  (Hz) 
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Fig.  29  Right  band-pass  filter  measured  transfer  characteristics  in  response  to  LED  chirp 
between  3  and  100  Hz:  magnitude  increases  20  dB/decade  and  phase  drops  from  -105°  to  - 
180°  (in  agreement  with  simulation  in  Fig.  31). 
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3.9.2  Sensor  vs.  Ground  Truth  Frequency  Characterization 

To  characterize  ocellar  sensor  frequency  response  with  respect  to  the  gyroscope 
(acting  as  the  ground  truth  velocity),  concatenated  sine  waves  were  given  as 
velocity  input.  The  operation  was  performed  in  the  valid  angle  range  previously 
described.  The  ocelli  output  is  a  voltage  value.  Its  magnitude  is  scaled  to  match 
with  the  gyroscope.  This  frequency  response  is  from  gyroscope  as  input,  to  the  full 
ocellar  sensor  as  output.  Within  the  test  frequencies,  the  ocellar  sensor  shows  a 
relatively  stable  magnitude  around  0  dB.  The  phase  response  is  degrading  over  the 
frequency  range.  Figure  30  shows  the  ocellar  sensor  frequency  response.  From  0.1 
to  1  Hz,  there  is  almost  no  phase  delay  between  ocelli  and  gyro.  After  1  Hz,  ocelli 
shows  a  phase  delay  of  approximately  15°.  The  data  after  10  Hz  were  not  taken  into 
account,  since  the  input  stimulus  cannot  exceed  10  Hz.  The  components  at  higher 
frequencies  are  due  to  mechanical  noise  inherent  in  the  motor.  Coherence  dips  are 
happening  at  the  same  frequencies  with  the  other  experiments;  therefore,  it  is 
believed  that  the  motor  was  not  able  to  implement  those  frequency  components. 
Ocelli  always  shows  a  coherence  above  0.5,  which  is  practically  acceptable. 
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Fig.  30  Ocelli  frequency  response  with  respect  to  gyro  as  input:  Frequencies  after  10  Hz 
were  shown  to  prove  the  decrease  in  coherence  out  of  controlled  motion  frequencies.  Ocellar 
magnitude  is  relatively  flat,  showing  around  ldB  difference  from  beginning  to  end.  Phase 
delay  reaches  to  -15°  at  10  Hz. 
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Figure  31  shows  the  time-domain  signals  comparing  gyro,  ocelli,  and  optic  flow  in 
0.5  Hz,  1  Hz,  5  Hz,  and  10  Hz  windows,  respectively.  The  outputs  of  optic  flow 
and  ocelli  are  scaled  to  match  gyro  output  (rad/s)  at  each  window.  The  ocelli  and 
optic  flow  are  in  agreement  with  gyro  signal  at  each  frequency  window. 


Fig.  31  Time  signals  of  gyro,  ocelli,  and  optic  flow  in  0.5-,  1-,  5-,  and  10-Hz  windows;  all 
sensor  outputs  are  scaled  to  match  gyro  (rad/s)  at  each  window 

Figure  32  shows  the  optic  flow  frequency  characterization  for  60  fps  data,  using 
Lucas-Kanade  with  25-pixel  window  size  and  16  feature  points  (4x4).  The  entire 
magnitude  response  is  very  close  to  flat.  There  is  almost  no  phase  delay  until  1  Hz, 
at  which  point  it  reaches  -35°  at  10  Hz.  Taking  into  account  that  these  optic-flow 
data  are  obtained  with  the  highest  frame  rate,  the  magnitude  response  is  expected 
to  degrade  as  the  frame  rate  decreases.  Although  the  optic  flow  frequency  response 
remains  approximately  flat  with  these  settings,  the  frequency  response  is  related  to 
the  motion  algorithm,  frame  rate,  and  window  size,  as  will  be  seen  next.  Varying 
these  parameters,  a  worse  high-frequency  response  will  be  obtained,  which  is  the 
real-life  case  with  flying  vehicle  onboard  computers.  All  in  all,  these  data  show  the 
maximum  bandwidth  the  optic  flow  can  achieve  within  the  test  limitations. 
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Fig.  32  Frequency  response  of  optic  flow  with  respect  to  gyro  as  input:  Overall  magnitude 
decrease  is  1.42  dB.  Phase  delay  reaches  to  -35°  at  10  Hz. 


3.10  Ocellar  Sensor-Gyro  Voltage-Velocity  Mapping 

To  understand  the  expected  ocelli  output  (in  volts)  for  a  given  gyro  (rad/s)  value, 
the  data  for  ocelli  and  gyro  across  1-10  Hz  test  frequencies  were  combined.  With 
0.005  rad/s  intervals,  expected  ocelli  output  (V)  and  gyro  (rad/s)  values  were 
calculated  (16  data  points  for  both).  These  points  were  fitted  to  a  line  of  equation 

/(x)  =  pi  *  x  +  p2  (44) 

where  pi  =  0.164  and  p2  =  2.489,  R2  =  0.9902.  Figure  33  shows  the  mapping 
plot.  It  shows  that  the  ocelli  output  is  monotonically  increasing  with  increasing  gyro 
amplitude,  implying  linearity.  The  standard  deviation  (error  bars)  is  large  because 
of  the  discretization  limit  of  the  ADC.  The  ADC  has  4.8  mV  resolution,  and  the 
overall  motion  is  within  140  mV. 
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Fig.  33  Ocelli-gyro  mapping  plot  shows  the  expected  ocelli  output  (V)  for  a  given  gyro 
measurement  (rad/s).  Ocelli  output  is  monotonically  increasing  with  increasing  gyro  values. 


3.11  Performance-Related  Parameters 


3.11.1  Frame  Rate 

Frame  rate  is  the  frequency  at  which  the  camera  displays  consecutive  images.  From 
Eq.  2,  the  Lucas-Kanade  algorithm  extensively  uses  spatial  and  temporal 
derivatives,  using  numerical  differentiation.  To  give  an  example,  let  f  be  a  given 
function  that  is  only  known  at  a  number  of  isolated  points.  The  problem  of 
numerical  differentiation  is  to  compute  an  approximation  of  the  derivative  f  of  f 
by  suitable  combinations  of  the  known  values  of  f. 

Assuming  that  function  f  is  differentiable,  the  derivative  f  (a)  for  some  real  number, 
a,  is  defined  as 


/'(a)  =  lta  ( -rjsiatm ) 
For  very  small  h,  this  derivative  can  be  approximated  by 


(45) 


(46) 
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This  approximation  involves  error,  and  this  error  increases  as  h  increases.  To 
demonstrate  this,  one  can  use  f(x)  =  sinx,  f  (x)  =  cosx  and  compute  the  error 

f  (x + fx) — f  (oc) 

between  f  (x)  =  cosx  and  f  (x) - - - .  The  error  will  increase  as  the  h  values 

increase. 


Figure  34  shows  the  frequency  response  of  60,  30,  and  20  fps  optic  flow  results 
from  gyro  as  input  and  from  optic  flow  as  output.  As  frame  rate  decreases,  optic 
flow  magnitude  response  rolls  off  steeper  and  reaches  to  -2.64,  -4.92,  and  -9.62 
dB  at  10  Hz  for  60,  30,  and  20  fps,  respectively.  For  all  frame  rates,  the  phase  delay 
remains  constant.  For  0. 1-1.1  Hz,  there  is  almost  no  phase  delay  between  optic 
flow  and  gyroscope.  After  1.1  through  10  Hz,  the  phase  delay  increases  and  reaches 
-35°.  The  coherence  of  20  fps  measurement  is  the  worst,  as  increasing  frame  rate 
increases  the  coherence,  as  well. 
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Fig.  34  Optic  flow  frequency  response  with  different  frame  rates,  as  seen  by  input  gyro:  As 
the  frame  rate  decreases,  roll-off  at  higher  frequencies  is  steeper.  Higher  frame  rate  results  in 
better  coherence.  Phase  delay  does  not  change  due  to  frame  rate. 

Besides  numerical  differentiation  error,  another  explanation  lies  under  the  Taylor 
series  approximation  used  to  derive  the  motion  constraint  equation.  Taking  only  the 
first-order  components  assumes  that  the  change  in  motion  is  small.  However,  when 
the  change  is  larger,  the  second-order  components  will  come  into  play  and  the 
motion  constraint  equation  will  no  longer  hold.  When  the  motion  is  too  fast  for  a 
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given  frame  rate,  the  spatial/temporal  estimate  assumption  breaks  down.  In 
practice,  this  resulted  in  an  optical  flow  measurement  of  erroneously  low 
magnitude. 

Aliasing  is  another  way  to  look  at  this  roll-off.  When  the  frame  rate  decreases, 
there  are  less  optic  flow  vectors  to  sample  the  given  sine  wave.  These  vectors  may 
be  computed  at  random  points  of  the  sine  wave,  not  exactly  catching  the  peak 
amplitudes.  If  the  frame  rate  is  higher,  more  optic  flow  points  will  result  in  a  more 
accurate  sine  wave,  catching  the  peaks. 

Scheider  et  al.55  study  the  optic  flow  outputs  as  the  angular  rate  changes.  According 
to  their  findings  for  2  different  optic  flow  algorithms,  optic  flow  matches  with  real 
rate  for  slow  motions  for  a  specific  resolution.  As  the  rate  increases,  optic  flow 
cannot  capture  images  often  enough  to  get  an  accurate  estimate  of  angular  rate. 
Optic  flow  first  draws  a  unity  line  with  real  rate,  then  this  line  starts  showing  a  fixed 
rate;  finally,  it  rolls  completely  off  to  zero. 

To  overcome  this,  one  may  increase  the  frame  rate,  or,  use  a  smaller  image  (e.g., 
binned  by  2)  to  double  the  frame  rate.  With  the  current  setup,  a  752  x  480-pixel 
image  can  go  up  to  20  fps.  When  the  image  is  binned  by  2,  the  frame  rate  increases 
to  60  fps  for  a  376  x  240-pixel  image.  This  solution  will  result  in  losing  maximum 
image  resolution. 

3.11.2  Window  Size 

The  Lucas-Kanade  algorithm  assumes  that  the  motion  is  the  same  for  all  pixels  in 
a  window  of  w  by  w  pixels.  This  tracking  window  size  determines  the  number  of 
equations  (hence,  optic  flow  vector  candidates)  to  be  used  in  the  least  squares 
method.  Assuming  constancy  in  motion,  more  optic  flow  vectors  will  give  more 
data  points  to  determine  the  best  fit  for  optic  flow.  However,  if  the  window  is  too 
large,  a  point  may  not  move  like  its  neighbors. 

Figures  35  and  36  show  the  window  size  versus  the  optic  flow  frequency  response. 
Changing  window  sizes  significantly  affected  the  coherence  plot.  Phase  delay 
remained  the  same  for  all  window  sizes.  Magnitude  is  the  most  erroneous  for  the 
10-  and  20-pixel  (10  x  10,  20  x  20)  window  sizes;  however,  it  stays  relatively  the 
same  for  the  rest.  The  coherence  is  the  worst  using  10  pixels.  It  improves  as  the 
window  size  increases  from  20  to  40  pixels,  and  remains  the  same  after  40  pixels 
through  70  pixels. 
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Fig.  35  Optic  flow  frequency  response  with  different  window  sizes  (w  =  10,  20,  30,  40),  as 
seen  by  input  gyro:  Very  small  windows  (10  x  10  pixel)  result  in  erroneous  magnitude 
response.  Magnitude  response  and  coherence  improve  as  window  size  increases,  phase  delay 
remains  the  same. 
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Fig.  36  Optic  flow  frequency  response  with  different  window  sizes  (w  =  50,  60,  70),  as  seen 
by  input  gyro:  After  50  x  50-pixel  window,  magnitude,  phase,  and  coherence  plots  do  not 
change. 
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3.11.3  Feature  Points 


Feature  points  are  the  number  of  center  pixels  located  on  each  image.  Around  these 
center  pixels,  optic  flow  vectors  are  calculated  within  the  window  size.  The  number 
of  feature  points  determines  the  number  of  optic  flow  vectors  computed.  The 
plotted  optic  flow  is  only  the  x  component  of  average  optic  flow  field.  The  feature 
points  are  equally  distributed  over  x  and  y  dimensions  of  the  image.  The  spacing 
between  them  is 


x  (or  y)spacing  = 


image  width(or  height ) 
number  of  feature  points  along  x  ( or  y) 


(47) 


where  x  feature  points  are  referred  to  as  x-by-x  center  pixels.  Figures  37  and  38 
show  the  image  scene  with  10  x  10  and  4x4  feature  points,  respectively. 


Fig.  37  Camera  scene  (10  x  10  feature  points) 


Fig.  38  Camera  scene  (4x4  feature  points) 
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Figure  39  shows  the  optic  flow  frequency  response  with  respect  to  different  number 
of  future  points.  The  2x2  feature  points  result  in  the  most  erroneous  optic  flow 
magnitude.  The  magnitude  response  improves  after  2x2  and  stays  relatively  the 
same  from  4x4  to  15x15  feature  points.  Similar  to  window  size  result,  the  change 
in  phase  remains  the  same  between  feature  points.  The  coherence  is  the  worst  using 
2x2  feature  points.  Increasing  the  feature  points  improves  the  coherence;  however, 
coherence  remains  the  same  after  8  feature  points.  This  means  that  a  sufficient 
number  of  optic  flow  vector  data  points  are  accumulated  to  make  the  best  fit  for 
optic  flow  with  8x8  feature  points.  More  feature  points  bring  redundant  data 
points. 


Frequency  (Hz) 


Fig.  39  Optic  flow  frequency  response  with  different  number  of  feature  points  (f),  as  seen  by 
input  gyro:  2x2  feature  points  result  in  erroneous  magnitude  plot.  As  the  feature  points 
increase,  magnitude  and  phase  plots  do  not  show  much  change;  however,  coherence  improves. 


3.11.4  Luminance  intensity 

DC  light  input  power  is  varied  to  understand  how  the  luminance  intensity  changes 
the  ocellar  circuit  output.  The  circuit  outputs  at  the  same  frequency  were  compared. 
Figure  40  shows  the  peak-to-peak  amplitudes  with  respect  to  input  power,  for  10 
Hz  motion.  As  the  power  increases,  amplitude  increases,  as  expected.  The  fitted 
line  has  coefficients  of  pi  =  0.088,  p2  =  0.006.  This  brings  a  necessity  for  “adaptive 
gain”  for  different  luminance  values  in  the  environment. 
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Light  source  input  power  (W) 


Fig.  40  Light  source  input  power  vs.  ocelli  peak-to-peak  amplitude:  Luminance  increase 
linearly  increases  the  peak-to-peak  amplitude.  DC  light  source  is  specified  in  Table  3. 

3.11.5  Photodiode  Bending 

The  photodiodes  should  be  bent  towards  the  light  source  to  share  an  intersecting 
FOV  and  to  satisfy  that  one’s  output  is  increasing  while  the  other’s  is  decreasing. 
The  bending  determines  how  much  their  FOVs  intersect  and  how  much  they  are 
seeing  the  edges  of  the  light  source  as  2  different  horizons.  The  reference  for 
bending  is  seen  in  Fig.  41.  If  P  =  0°,  no  common  luminance  is  shared.  If  P  =  90°, 
their  FOVs  completely  intersect  and  no  symmetric  change  with  respect  to  each 
other  is  observed.  Assumption  is  satisfied  for  P  values  between  0  and  45°, 
specifically  P  =  30°,  40°,  45°  raw  outputs  are  observed  to  be  symmetric  to  each 
other.  For  P  -  90°,  the  same  sine  wave  shape  is  seen  at  the  same  instant. 


Approved  for  public  release;  distribution  is  unlimited. 

42 


Fig.  41  Bending  illustration:  The  photodiodes  should  share  an  intersecting  FOV  toward  the 
light  source  for  the  sensor  to  operate.  Bending  values  30°  <  P  <  45°  were  observed  to  give 
symmetric  photodiode  outputs.  B  =  90°  completely  overlaps  the  FOVs,  without  distinct 
horizons  for  each  photodiode. 

3.12  Test  Setup  Limitations 

The  maximum  motion  frequency  achieved  with  the  motor  is  around  10  Hz  for 
velocity  mode.  For  position  mode,  the  frequency  is  even  lower,  at  2  Hz.  Above 
these  frequencies  for  related  modes,  the  motor  does  not  follow  the  input 
position/velocity.  The  higher  frequency  components  in  the  plots  are  from  inherent 
mechanical  vibrations  of  the  motor  and  the  flickering  of  laboratory  lights  at  60  Hz 
or  its  harmonics  at  120-1 80Hz.  The  camera  used  has  a  theoretical  claim  of  87  fps 
frame  rate.  However,  when  this  frame  rate  is  used,  frame  drops  are  observed.  Optic 
flow  calculation  is  highly  corrupted  by  frame  drops.  Frame  drops  were  minimized 
with  60  fps  frame  rate. 

Also,  using  2  cameras  for  covering  more  field  was  our  first  attempt.  This 
configuration  needs  triggering  to  satisfy  that  the  cameras  are  taking  photo  at  the 
same  time.  Triggering  was  achieved  with  2  PX-4  Inertial  Measurement  Units 
working  as  master  and  slave.  However,  data-transfer  limitation  from  USB  port 
reintroduced  the  lower  frame  rate  problem;  hence,  frame  drops  stopped  after 
switching  to  one  camera  at  the  same  frame  rate.  To  allow  for  triggering,  frame  rate 
had  to  be  decreased  to  20  fps,  which  limited  the  optic  flow  bandwidth.  To  show  the 
maximum  achievable  bandwidth  for  optic  flow  with  this  configuration,  only  one 
camera  is  used  with  60  fps,  376  x  240  pixels  image. 
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4.  Sensor  Fusion 


As  described  in  Section  3,  ocellar  sensor  shows  a  relatively  stable  magnitude  across 
the  test  frequencies.  Optic  flow  frequency  response  can  keep  up  with  the  ocellar 
sensor  for  60  fps  data.  As  the  frame  rate  decreases,  optic  flow  magnitude  plot  rolls 
off.  At  high  motion  frequencies  (where  the  optic  flow  information  degrades),  it  is 
possible  to  use  the  ocellar  circuit.  This  section  first  presents  the  biological 
background  for  sensor  fusion  in  insect  compound  eyes  and  the  ocelli.  Then  the 
general  fusion  approaches  from  the  literature  are  discussed.  Finally,  the  optic  flow 
and  the  ocellar  sensor  data  are  fused  to  demonstrate  the  high-frequency  roll-off 
compensation  of  optic  flow,  using  the  ocellar  sensor. 

4.1  Biological  Background  for  Sensor  Fusion 

From  the  behavioral  studies,  ocelli  and  compound  eyes  are  thought  to  work  together 
for  flight  stabilization  abilities.21  22  24  In  the  blowfly,  it  is  previously  studied  that 
lobula  plate  tangential  cells  estimate  the  self-motion  by  taking  local  motion 
information  from  compound  eyes.  One  of  the  cells  that  are  reported  to  respond  optic 
flow  information  is  a  tangential  cell,  called  VI.56  Parsons  et  al.57  reports  that  VI 
responds  to  ocelli  stimulus  as  well.  The  response  increases  with  the  rate  at  which 
the  light  intensity  changes,  implying  that  VI  might  be  encoding  angular  velocity 
information,  as  well  as  optic  flow  information.  Haag  et  al.58  experiments  that  a 
prominent  descending  neuron  called  DNOVS1  receive  input  from  2  sources — from 
the  photoreceptors  of  compound  eye  via  large-field  motion  sensitive  cells  and  from 
photoreceptors  of  ocelli  via  ocellar  intemeurons.  Parsons  et  al.59  reports  that  lobula 
plate  neurons  combine  inputs  from  both  ocelli  and  compound  eyes.  Ocellar 
responses  encode  information  in  3  axes,  whereas  compound  eyes  encode  in  9.  This 
reveals  that  ocelli  are  only  able  to  detect  rotation  around  3  axes,  thus  offering  less 
specificity  with  respect  to  compound  eye.  If  we  assume  a  direct  summation  of  ocelli 
and  compound  eye  neuronal  signals,  this  might  help  the  flight  behavior  in  3  axes 
(since  there  will  be  more  information  for  3  axes,  from  both  compound  eyes  and 
ocelli).  However,  for  the  other  6  axes,  ocelli  might  output  “zero”  and  the  fused 
response  from  both  compound  eye  and  ocelli  might  degrade  the  flight  behavior, 
which  seems  like  counterintuitive.  Parsons  et  al.59  suggest  that  each  VS  neuron  is 
tuned  to  the  ocellar  axis  closest  to  its  compound  eye  axis,  combining  the  speed  of 
ocelli  with  the  accuracy  of  compound  eyes  without  compromising  either. 

Having  said  that  ocelli  are  faster  than  compound  eyes,  what  is  the  quantitative 
difference  between  these  latencies?  The  response  latency  depends  strongly  on 
experimental  parameters,  such  as  contrast  and  frequency  of  a  moving  stimulus.  For 
example,  with  increasing  contrast  and  high  frequency,  latency  decreases. 
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Moreover,  temperature  changes  and  the  age  of  the  fly  affect  the  latency.60  Safran  et 
al.61  report  that  motion  sensitive  neuron  HI  (compound  eye  neuron)  transmits 
signals  in  20-30  ms.  Parsons  et  al.59  measures  6  ms  for  ocellar  latency,  which 
indicates  a  significant  reduction  when  compared  to  compound  eyes.  For  high 
frequency  disturbances,  low-latency  ocellar  neurons  will  be  needed. 

4.2  Fusion  Approaches 

In  motion  detection  and  control  systems,  especially  in  flight  control  and  inertial 
navigation,  different  kinds  of  sensors  are  used  on  one  platform.  When  measuring  a 
particular  variable,  a  single  type  of  sensor  may  not  be  able  to  meet  all  the  required 
performance  specifications.  For  example,  both  accelerometer  and  gyroscope  data 
can  be  used  to  compute  angles.  Since  an  accelerometer  gives  acceleration,  angles 
can  be  reconstructed  from  accelerometer  output  by  2-fold  integration.  Similarly,  a 
gyroscope  gives  velocity  information  and  one  integration  would  be  enough.  The 
accelerometer  is  known  to  be  good  for  “long  term”,  meaning  that  it  does  not  drift. 
A  gyroscope  is  good  for  “short  term”;  it  is  known  to  have  poor  drift  characteristics 
but  is  able  to  give  a  fast  response.  An  ideal  combination  would  be  a  fast  transient 
response  with  no  drift,  by  combining  good  qualities  from  2  measurements. 

Theoretically,  if  a  time-varying  signal  is  applied  to  both  a  low-pass  and  high-pass 
filter  with  unity  gain,  the  sum  of  the  filtered  signals  should  be  identical  to  the  input 
signal.  (See  Fig.  42.)  Assume  that  x  and  y  are  noisy  measurements  of  some  signal 
z,  with  x  employing  low-frequency  noise  and  y  employing  high-frequency  noise. 
Z’  is  the  estimate  of  the  signal  z  produced  by  the  complementary  filter. 


Fig.  42  Illustration  of  complementary  filter 

Practically,  complementary  and  Kalman  filters  provide  the  fusion  of  2  signals.  The 
Kalman  filter,  working  in  time  domain,  needs  statistical  description  of  the  noise 
corrupting  the  signals.  This  noise  is  assumed  to  be  Gaussian  white  noise. 
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Complementary  filters  approach  the  problem  from  the  frequency  domain,  and  they 
are  generally  used  for  the  fusion  systems  that  do  not  deal  with  noise.  For  digital 
implementation,  the  complementary  filter  has  considerable  advantage  over  the 
Kalman  filter,  as  Kalman  gains  are  not  computed  for  each  state.  Therefore,  after 
determining  the  filter  coefficients  for  complementary  filter,  the  update  rate  of 
complementary  filter  can  be  higher  than  Kalman  filter  for  each  loop.  This  is  an 
important  consideration  for  the  applications  in  which  high-rate  loop  closure  is 
necessary. 

4.3  Previous  Sensor  Fusion  Implementations 

Sensor  fusion  is  governed  by  complementary  and  Kalman  filtering  in  the  literature, 
generally  for  virtual  reality  applications  in  computer  vision  and  attitude  control. 
Vision-based  information  helps  avoiding  the  errors  resulting  from  integrating  the 
inertial  sensors  over  time.  Vaganay  et  al.62  fuse  2  accelerometers  and  3  gyroscopes 
for  an  indoor  mobile  robot  to  obtain  attitude  information  using  an  extended  Kalman 
filter.  Foxlin63  fuses  gyroscopes  and  inclinometer  for  head-tracking  using  a  Kalman 
filter.  You  and  Neumann64  integrate  high-frequency  stable  gyroscope  and  low- 
frequency  stable  vision-based  tracking  using  a  Kalman  filter  for  an  augmented 
reality.  Wu  et  al.65  use  an  extended  Kalman  filter  that  takes  information  from 
camera  images,  inertial  measurement  unit,  and  magnetometers  to  estimate  the  pose 
of  the  vehicle.  Cheviron  et  al.66  fuse  accelerometer,  gyroscope,  and  vision  sensors 
to  obtain  position,  velocity,  and  attitude  information  for  an  unmanned  aerial  vehicle 
(UAV),  using  a  nonlinear  complementary  filter  framework.  Bleser  and  Strieker67 
use  an  extended  Kalman  filter  to  fuse  vision-based  output  for  slow  movements  and 
inertial  sensor  output  for  fast  movements  for  virtual  reality  applications.  Conte  and 
Doherty68  use  a  Kalman  filter  to  fuse  data  from  3  accelerometers  and  3  gyroscopes 
with  a  position  sensor  for  UAV  navigation.  Position  sensor  input  is  either  from  a 
global  positioning  system  (GPS),  when  GPS  is  available,  or  from  vision  system 
(feature  tracking)  when  GPS  is  not  available.  Schall  et  al.69  fuse  GPS  data,  inertial 
sensor  data,  and  camera  image  data  for  global  pose  information  for  augmented 
reality.  Inertial  and  GPS  data  are  fused  using  a  Kalman  filter.  Achtelik  and  Weiss70 
use  an  extended  Kalman  filter  to  fuse  air  pressure  sensor  and  vision  framework 
(computationally  expensive)  with  inertial  sensor  data  to  handle  the  fast  movements 
and  disturbances  of  the  micro-air  vehicle.  Campolo  et  al.71  propose  a 
complementary  filter  to  fuse  magnetometer,  accelerometer,  and  gyroscope  data  for 
attitude  estimation.  In  this  section,  the  time  domain  signal  of  both  ocellar  and  optic 
flow  outputs  are  combined  to  extend  the  optic  flow  frequency  response. 
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4.4  Ocellar  Sensor-Optic  Flow  Fusion  Approach 


Previous  results  show  that  optic  flow  shows  roll-off  at  high  frequency  motion.  On 
the  other  hand,  the  ocellar  sensor  shows  a  relatively  flat  response  at  high 
frequencies.  This  experiment  uses  a  camera  capable  of  87  fps,  in  theory,  and  a 
high-speed  Ubuntu  laptop.  Even  with  this  configuration,  the  real  frame  rate 
obtained  from  the  camera  becomes  60  fps  because  of  data  transfer  limitations  of 
USB  busses.  The  frame  rates  higher  than  60  fps  result  in  dropped  frames  and 
corrupt  the  optic  flow  output. 

Commercially  available  single-board  computers  (e.g.,  Raspberry  Pi71)  allow  for 
lower  frame  rates.  Practically,  Raspberry  Pi  2  is  limited  to  15-20  fps  for  the  same 
Lucas-Kanade  algorithm  used  in  this  experiment.  A  more  expensive  model,  Odroid 
XU4,72  is  capable  of  60  fps;  however,  its  cost  doubles  Raspberry  Pi  ($75  vs.  $35). 
A  relatively  cheap  single-board  computer  will  have  a  limited  optic  flow 
computation  bandwidth.  On  the  other  hand,  the  ocellar  sensor  offers  a  fast,  cheap, 
and  low-power  alternative  to  optic  flow  computation.  It  has  a  relatively  flat 
magnitude  and  phase  response,  and  it  is  an  attractive  alternative  for  rotational 
motion.  However,  its  performance  is  highly  dependent  on  the  luminance.  It  assumes 
a  constant  luminance  gradient  from  sky  to  ground.  Optic  flow  computation  does 
not  have  such  an  assumption,  and  it  only  needs  a  texture  around  it.  Moreover,  this 
setup  uses  a  180°-FOV  lens  to  obtain  wide-field  motion.  To  have  more  FOV,  the 
number  of  cameras  may  be  increased;  this  will,  however,  create  the  necessity  for 
simultaneous  triggering  of  the  2  cameras.  When  this  setup  is  used  with  2  cameras, 
the  triggering  reduces  the  camera  frame  rates  down  to  20  fps.  This  reduction  is 
expected  to  be  more  using  a  cheaper  configuration.  Fower  frame  rate  will  introduce 
a  lower  optic  flow  bandwidth,  making  the  optic  flow  sensing  incapable  of 
performing  at  high  frequencies.  To  compensate  for  the  high-frequency  roll-off  of 
optic  flow,  ocellar  sensor  data  are  fused  with  optic  flow.  Figure  43  shows  the 
complementary  fusion.  The  optic-flow  data  are  low-pass  filtered  with  a  fourth- 
order  Butterworth  filter.  The  inverse  of  this  filter,  a  fourth-order  Butterworth  high- 
pass  filter,  with  the  same  cutoff  frequency  is  used  to  high-pass  filter  the  ocelli  data. 
The  reason  for  using  a  fourth-order  Butterworth  filter  instead  of  single-pole  high- 
and  low-pass  filters  is  that  it  resulted  in  a  better  coherence.  Single-pole  filter 
combinations  decreased  the  coherence  values  at  high  frequencies.  The  fusion 
operation  increased  the  bandwidth  and  decreased  the  phase  delay  of  optic  flow. 
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Fig.  43  Frequency  response  ocelli,  optic  flow,  and  their  complementary  fusion:  Fourth- 
order  Butterworth  filter  was  used  to  high-pass  ocelli  and  low-pass  optic  flow.  The  normalized 
cutoff  frequency  had  to  be  kept  very  small  to  make  use  of  ocelli’s  relatively  flat  magnitude  and 
less-delayed  phase.  Fused  response  shows  coherence  is  better  than  optic  flows. 

An  even  more  direct  way  is  taking  the  weighted  average  of  optic  flow  and  ocelli. 
While  this  approach  will  not  provide  fully  low-pass  filtered  optic  flow  and  high- 
pass  filtered  ocelli,  if  the  ocelli  weight  is  kept  high,  the  result  will  be  very  similar 
to  optic  flow.  Figure  44  shows  another  fusion  that  implements 

a  *  ocelli  +  (1  —  a)  *  optic  flow  (48) 

where  a  =  0.9.  Magnitude,  phase,  and  coherence  plots  result  in-between  ocelli  and 
optic  flow,  very  close  to  ocelli. 
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Fig.  44  Frequency  response  ocelli,  optic  flow,  and  their  weighted-average  fusion:  Ocelli  and 
optic  flow  time-domain  signals  are  combined  to  obtain  a  result  close  to  ocelli. 


However,  complementing  both  ocelli  and  optic  flow  readings  gives  a  result  close 
to  ocelli.  It  assumes  that  the  ambient  luminance  distribution  is  as  calibrated  in  this 
experiment.  We  know  that  the  ocelli  magnitude  increases  with  increasing 
luminance.  This  peak-to-peak  amplitude  is  a  linear  function  of  input  light  source 
power,  as  seen  in  Section  3.  Additionally,  ocellar  sensor  has  to  be  in  a  “valid  range”. 
All  in  all,  optic  flow  is  immune  to  luminance  intensity.  It  gives  a  flat  magnitude 
response  at  low  frequencies.  Ocelli,  however,  is  vulnerable  to  luminance  intensity 
and  it  does  not  show  a  roll-off  in  magnitude  as  optic  flow  shows.  It  would  be  ideal 
to  combine  the  good  properties  of  both  measurements  real-time.  Ocelli  magnitude 
plot  with  respect  to  increasing  luminance  intensity  should  result  in  a  plot  like  Fig. 
45.  Optic  flow  magnitude  plot  shows  low-pass  characteristics. 
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Ocelli  Magnitude  Response  with  Luminance  Change  (L(n+1)>L(n)) 
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Fig.  45  Magnitude  response  of  ocelli  with  different  luminance  values  and  optic  flow  at  30 
fps:  Increasing  luminance  implies  higher  magnitude  for  ocelli  IL1<L2<L3<  L-i<  Ls).  Ambient 
luminance  change  brings  adaptive  gain  necessity.  Upper  figure  is  the  magnitude-scaled 
versions  of  ocelli  response,  not  derived  from  real  luminance  values. 

A  mechanism  that  allows  for  switching  from  one  mode  to  another  is  desired  to 
decide  which  sensor  to  use.  This  switching  mechanism  may  be  a  gyroscope.  The 
gain  adjustment  may  be  done  with  a  feedback  from  ocelli  output  that  is 
continuously  compared  with  gyroscope/optic  flow  output.  If  a  valid  region  for 
ocelli  is  found,  ocelli  is  preferred  over  optic  flow  due  to  its  high  speed.  A 
hypothetical  iterative  approach  is  shown  in  Fig.  46.  First,  ocelli  gain  is  adjusted 
with  the  use  of  a  lookup  table  and  the  error  between  ocelli  and  gyro  is  computed.  If 
this  error  is  below  a  threshold,  the  gain  adjustment  is  satisfied.  After  this,  the 
validity  of  ocelli  output  is  confirmed  by  computing  the  error  between  the  gyro  and 
optic  flow.  If  these  comparisons  allow,  ocelli  is  preferred  to  be  used  for  closed-loop 
rate  stabilization.  If  not,  either  gyro  or  optic  flow  is  used.  Ocelli  gain  can  be  adjusted 
with  digital  potentiometers  and  an  operational  amplifier.  The  digital  potentiometers 
are  controlled  from  microcontroller.  For  a  gain  less  than  1,  a  voltage  divider  reduces 
the  ocelli  output.  For  a  gain  greater  than  1,  a  noninverting  amplifier  increases  the 
ocelli  output.  This  output  is  continuously  fed  back  to  the  microcontroller  to 
compare  the  gyroscope  and  ocelli  error  to  find  a  new  gain  value  from  the  lookup 
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table  and  adjust  the  potentiometers  accordingly.  Figure  47  shows  the  possible 
circuit  configuration  with  microcontroller. 


Adjust  ocelli  gain  to  match  with 
gyroscope/OF  output 
(Lookup  table  or  binary  search). 

If  ocelli<gyro/OF  “^Vhighgam 

If  ocelli>gyro/OF  “^Viowgain 


Check  if  ocelli  is  valid. 

^  ^gyro/OF  ^ ocelli 

If  e<threshold 

Fuse  ocelli  and  optic  flow. 
OR 

Turn  off  optic  flow. 

Repeat  in  interrupt  statement. 


Fig.  46  Hypothetical  sensor  decision  approach:  Adjust  ocelli  gain  by  continuously 
computing  error  between  gyro/OF  and  ocelli;  check  if  ocelli  is  valid  to  use  by  comparing 
gyro/OF;  use  ocelli  if  comparisons  allow. 
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Fig.  47  Hypothetical  ocelli  gain  adjustment  approach:  Gains  >  1  are  tuned  by  noninverting 
op-amp.  Gains  <  1  are  tuned  by  voltage  divider.  The  tuned  outputs  are  compared  with  lookup 
table  and  microcontrollers  iteratively  tune  the  digital  potentiometers  until  error  threshold  is 
low  enough. 

5.  Conclusion  and  Future  Work 


5.1  Conclusion 

Frequency-domain  characterization  of  optic  flow  and  ocellar  sensors  are  presented. 

The  advantages  and  disadvantages  for  both  sensing  mechanisms  are  discussed.  In 

summary: 

•  Ocellar  sensor  shows  a  relatively  flat  magnitude  response  and  less  phase 
delay  than  optic  flow. 

•  Ocellar  sensor  is  attractive  for  high-rate  loop  closure  since  it  is  cheaper  and 
faster  than  high-quality  cameras. 

•  The  displacement  dynamic  range  of  the  ocellar  sensor  is  observed  to  be  1 
radian  with  this  setup,  due  to  the  small  size  of  the  light  source.  Using  a 
larger  light  source,  higher  displacements  may  be  achieved. 

•  The  frequency  dynamic  range  of  ocellar  sensor  is  observed  to  be  up  to 
10  Hz  with  motion  and  up  to  low-frequency  cutoff  without  motion.  Ten 

Approved  for  public  release;  distribution  is  unlimited. 


52 


hertz  is  a  limitation  from  mechanical  test  setup;  higher  motion  frequencies 
are  expected  due  to  the  circuit  simulation  and  LED  experiment  results.  For 
outdoor  experiments,  the  low-frequency  cutoff  of  the  band-pass  circuit  can 
be  eliminated  since  there  is  no  flickering  issue  outdoors. 

•  Ocellar  sensor  magnitude  shows  a  linear  relationship  with  luminance 
intensity.  Since  it  is  highly  luminance-dependent,  an  adaptive  gain 
calibration  is  necessary  for  usage  with  different  luminance  levels. 

•  Ocellar  sensor  shows  monotonic  increase  with  increasing  gyro  values. 

•  Optic  flow  magnitude  rolls  off  at  high  frequencies.  Specifically,  60  fps  can 
keep  up  with  ocelli  response.  Thirty  and  20  fps  show  roll-off  at  7  Hz.  Less 
frame  rate  shows  steeper  roll-off.  Phase  delay  increases  with  increasing 
frequency.  All  frame  rates  tested  show  the  same  phase  delay  across  all 
frequencies. 

•  Optic  flow  algorithm  parameters  (feature  points,  window  size)  affect  the 
coherence.  No  significant  change  in  magnitude  and  phase  plots  is  observed, 
except  for  erroneous  magnitudes  for  extremely  small  window  sizes  or 
feature  points. 

5.2  Future  Work 

Several  potential  directions  may  be  taken  to  extend  the  work  of  this  report.  Taking 
the  characterization  results,  performance  parameters,  and  hypothetical  sensor 
fusion  suggestions  into  account,  a  closed-loop  optic  flow  and  ocellar-based  fusion 
may  be  implemented  to  perform  real-time  stabilization  and  disturbance  rejection. 
Multiple  ocellar  sensors  with  lenses  may  be  placed  in  an  array-like  fashion  on  a 
flying  vehicle  to  extend  the  current  FOV  of  the  ocellar  sensor.  The  outputs  of 
ocellar  sensor  may  be  matched  with  predefined  patterns  to  inform  where  exactly 
the  disturbance  occurs. 

The  combination  of  optic  flow  computations  and  ocellar  sensor  gives  both  slow  and 
fast  alternatives  for  horizon  detection  and  angular-rate  sensing. 

The  coherence  in  ocellar  sensor  and  optic  flow  frequency  response  plots  show  dips 
at  specific  frequencies.  The  reason  for  these  dips  could  not  be  identified  during  the 
experiments.  If  these  dips  were  caused  form  the  motor  mechanical  noise,  both  the 
gyroscope  and  the  ocellar  sensor  should  be  able  to  pick  the  mechanical  noise  up, 
resulting  in  the  same  motion  for  both  of  them.  Also,  the  motor  resonance  and 
gyroscope  resonance  possibilities  have  been  eliminated  after  confirming  the  time- 
domain  signals  with  the  ground  truth.  It  is  presumed  that  the  ocellar  sensor  may  be 
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slightly  modulating  the  input  sine  wave  at  these  frequencies.  The  experiments  can 
be  repeated  by  using  another  ocellar  sensor  board  and/or  another  motor. 

To  compensate  for  the  optic  flow’s  slow  rate,  another  direction  might  be  converting 
the  optic  flow-ocelli  system  to  a  fully  analog  scheme.  Combining  both  sensors  in 
analog  domain  might  give  the  complementary  approach  in  a  compact,  fast,  and 
lightweight  way.  While  digital  optic  flow  computation  has  the  freedom  of  easy 
adaptation  and  reconfiguration  with  different,  sophisticated,  and  robust  algorithms, 
subthreshold  analog  very-large-scale  integration  (VLSI)  optic  flow  designs  are 
much  smaller,  lightweight,  low  power,  and  faster.  One  may  argue  that  the 
one -board  computers  are  already  lightweight.  However,  decrease  in  size  and  weight 
are  extremely  important  factors  for  micro  aerial  vehicle  design.  VLSI  allows  the 
photodiodes  and  computation  circuitry  to  be  fabricated  on  a  piece  of  silicon; 
therefore,  it  is  very  suitable  for  vision-based  sensing. 

Insect  ocelli  have  high  responsivity  to  ultraviolet  wavelengths.  A  completely 
different  direction  might  be  taking  the  ocellar  sensor  outside,  using  the  sky-ground 
discrimination  in  ultraviolet  wavelengths.  The  wideband  photodiodes  in  current 
circuit  can  be  replaced  with  ultraviolet  photodiodes.  A  detail  to  consider  is  the 
ultraviolet  intensity  difference  in  sky  and  ground  in  different  times  of  the  day  and 
different  weather  conditions.  On  a  sunny  day,  the  results  show  that  sky  is  brighter 
than  ground  in  ultraviolet.  On  a  cloudy  day,  it  can  be  the  opposite.  While  the 
constant  and  negative  luminance  gradient  may  not  be  satisfied  for  all  cases,  specific 
weather  conditions  (e.g.,  sunny  day,  no  clouds)  can  allow  for  outdoor  use.  We  have 
built  the  ultraviolet  version  of  the  ocellar  sensor  and  tested  outside.  A  main  problem 
is  the  uneven  ultraviolet  intensity  coming  to  both  ultraviolet  photodiodes.  On 
cloudy  days,  there  is  nearly  no  ultraviolet  difference  between  sky  and  the  ground; 
it  is  thought  that  the  clouds  are  blocking  the  ultraviolet  portion  in  the  sunlight.  On 
sunny  days,  one  photodiode  should  not  see  the  high  intensity  created  by  the  sun  and 
should  only  have  the  portion  coming  from  the  sky.  The  instantaneous  displacement 
of  the  clouds  and  the  wind  are  also  factors  that  create  the  uneven  ultraviolet 
intensity  on  both  photodiodes.  While  these  cases  make  it  hard  to  test  outside,  we 
have  seen  with  some  datasets  that  it  agrees  with  gyroscope  output. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


2-D 

2-dimensional 

AC 

alternating  current 

ADC 

analog-to-digital  converter 

DC 

direct  current 

EMD 

elementary  motion  detector 

FOV 

field  of  view 

fps 

frames  per  second 

GPS 

global  positioning  system 

LED 

light-emitting  diode 

LSB 

least  significant  bytes 

MCU 

Microcontroller  Unit 

MSB 

most  significant  bytes 

SPI 

serial  peripheral  interface 

sUAS 

small  unmanned  aircraft  systems 

UAV 

unmanned  aerial  vehicle 

VLSI 

very-large-scale  integration 

Approved  for  public  release;  distribution  is  unlimited. 


61 


1  DEFENSE  TECHNICAL 
(PDF)  INFORMATION  CTR 

DTIC  OCA 

2  DIRECTOR 

(PDF)  US  ARMY  RESEARCH  LAB 
RDRL  CIO  L 

IMAL  HRA  MAIL  &  RECORDS 
MGMT 

1  GOVT  PRINTG  OFC 
(PDF)  AMALHOTRA 

1  US  ARMY  RESEARCH  LAB 
(PDF)  RDRL  SER  L 

JOSEPH  K  CONROY 

1  UNIV  OF  MD  OR  GA  TECH 
(PDF)  NIL  GUREL 

1  UNIV  OF  MD 
(PDF)  TIMOTHY  HORIUCHI 

1  UNIV  OF  COLORADO 
(PDF)  SEAN  HUMBERT 


Approved  for  public  release;  distribution  is  unlimited 


